Windows环境下Kafka与Zookeeper安装配置与启动过程

Windows环境下Kafka与Zookeeper安装配置与启动过程.软件下载下载jdk1.8.0_60下载zookeeper-3.4.9.tar.gz下载kafka_2.12-.tgz.环境准备安装配置java环境，配置jdk1.8为操作系统java运行时环境安装jdk1.8将网站下载的jdk1.8.0_60压缩包解压缩到c:\programfiles\java目录下，如下图：计算机►OS(C;)卜ProgramFiles►Java卜查看(V)工具⑴帮助(H)打开 包含至嶙中， 地享， 新建立件夹Pad 4 名称j2£dkl.4.0Decrypteridkl.6.026Makerjdkl.&.0_45iailjdkl-7,0_L7jdkl.3,0_60配置java环境如图在“控制面板--系统和安全--系统”页，鼠标点击“高级系统设置”，在弹出的“系统属性”页面选择高级tab页：

隼看M_L具⑴器蜘H]’隼看M_L具⑴器蜘H]’系统厘性控制面板主方S设g管理器＞谈蹭9系统《那1+1高级冢统设量点击“环境变量”按钮后，在弹出的环境变量对话框中点击“系统变量”栏中的“新建”按钮，在弹出的“新建系统变量”窗口中输入如下：

点确定确定后，再配置JRE_HOME

环境变皇系统娈量出）变重咱上CIASSPATH.;%JAVA_HONE%/Lih/di. ..=lCemSpecCTGMRElflC_ALTD?L^YUiXWincoys'1svD：em^i：',.c：jnd.exemsGLttyC Filet(iSO\Ct.'jiuiL^jiF…T新建（¥）...〔编辑（工|［删除（L）系统屋性I计算机名I硬件二I高级委保护I远程I_ I「环境变昌 22同时打开一个cmd窗口在其中输入：SETJAVA_HOME=C:\ProgramFiles\Java\jdk1.8.0_60SETJREHOME=%JAVAHOME%\jre

SETCLASSPATH=.;%JAA_HOME%/lib/dt.jar;%JAVA_HOME%/lib/tools.jar;%JRE_HOME%\lib\;%JRE_HOME%\lib\rt.jar;%JRE_HOME%\lib\jce.jar;%JRE_HOME%\lib\metadata-extractor-2.4.0-beta-1.jar;%JRE_HOME%\lib\mediautil-1.0.jar;SETPath=%JAVA_HOME%\bin;%JAVA_HOME%\jre\bin;在命令行输入java-version,输出如下，表明系统已启用java配置国管理员：C:\WindcM'sXsystem32\cmd.exe保留所有权利.MicrosoftWindows［版本6.1.T6G1］版权所有［c)2009MicrosoftCorporationD保留所有权利.C；\Users\liyongjiang>jaua-uersionjauaversion"18.0_G0"JauaTTH)SERuntimeEnuironment(build1.8.G_6G-b27)JauaHotSpotiTMj64-BitServerUM(build25.GG-b23,mixedmode)C：\Users\liyongjiar)9>.Zookeeper安装配置解压与目录设计在目录e:\apache\zookeeper\下解压文件zookeeper-3.4.9.tar.gz,并将解压文件夹改名为zk0,复制该文件夹到相同目录下分别改名为zk1、zk2.zkOzklAzk2在zk0\zk1\zk2目录下都创建空文件夹data和logs，如下图iconnib 2017/3/2114:15文件夹|1rlW苴2017/3/2216:03如院创建日两:2017/3/2214:29大小:2.65KB2017/3/2114:15如髅文件夹:version-22017/3/2114:15文件去文件myid2017/3/311^1:15文件夹IJlogs2017/3/2216:04文件夹3.2在data目录中创建文件myid，设置zookeeper服务器的序号。version-2myid2017/3您version-2myid2015/7；912:05Myid的内容分别为0、1、2 UL।ili।•"」iiiiii-io|配置zookeeper的属性配置zk0\zk1\zk2目录下的config/zoo.cfg文件。zk0/config/zoo.cfg文件内容为：tickTime=2000initLimit=5syncLimit=2dataDir=E:/Apache/zookeeper/zk0/datadataLogDir=E:/Apache/zookeeper/zk0/logsclientPort=2181server.0=:2888:3888server.1=:2889:3889server.2=:2890:3890zk1/config/zoo.cfg文件内容为：tickTime=2000initLimit=5syncLimit=2dataDir=E:/Apache/zookeeper/zk1/datadataLogDir=E:/Apache/zookeeper/zk1/logsclientPort=2182server.0=:2888:3888server.1=:2889:3889server.2=:2890:3890zk2/config/zoo.cfg文件内容为：tickTime=2000initLimit=5syncLimit=2dataDir=E:/Apache/zookeeper/zk2/datadataLogDir=E:/Apache/zookeeper/zk2/logsclientPort=2183server.0=:2888:3888server.1=:2889:3889server.2=:2890:3890启动zookeeperserver开三个cmd窗口，依次执行如下指令:E:cdE:\Apache\zookeeper\zk0\binzkServer.cmdE:cdE:\Apache\zookeeper\zk2\binzkServer.cmd

E:cdE:\Apache\zookeeper\zk1\binzkServer.cmd在三个窗口未同时启动前，前面开的窗口会报警:31管理员C;\Wind□wsXsystem32Umd.exe-zkServer.cmdatjaua.r?t.Ab5trectPLairSocketlmpl.doConnectfAbstractPlainSocketlmpl.jeua：350)atjaua.n&t.Ab$tr£ctPLairiSock«tImpl.connectToAddre5sfAb^tractPlainSocketImpl.jaua；2QS)atjaua.r&t.fibstractPlainSocketliiipl.cannect(flbstractPlainSockatImpl.jaua；1BS)atjaua.ret.Plainsocketlmpl.GcnnectfPLainSocketlnipl.jaua：1T2)atjflua.ret.SocksSocketImpl.connpct(SocksSocketImpl.jaua：392)atjflua.ret.Socket.connect(Socket.jaua：589)aterg.apache.zooke^p&r.seru&r.quoruiti.QuoruitiCnxManeg^r.conn^ctOnefQuoruitiCnxManag^r.jaua：381)ator^.spache.zookeeper.seruer.quorum.QuorumCnxManager.toSendi；QuorumCnxM.jaua；354〕aterg.apache.zoakeeper.ceruer.quorum.FastLpaderELectior$Messergpr$lilarkprScessfFastLead&rElection.jaua：M521aterg.apache.zookeeper.seruer.quorum.FastLcaderELeetior$Mp35prgcr$lilorkerSender.rum；FastL^aderElpction.jaua：433)atjaua.lang.Thread.ruri[Thread.jsub：7^5)2017-03-2410：O7：O3.649[itiyid：0]-INFO[Work^rSender[myid：0]：QucrumPc&r$QuoriJitiSeruer@143]-Resoluedhostname；toaddress；/2017-03-2410:07:10,004|吗idE]-WARM[011”川户€€1■[叫id二⑴/日；如U；13；口；口；◎；13；2181：FolLower@116]-GotzxidSsSOOOflOQOIespectedSkI201F-03-2H10：0r；ie.015[myid；e]-INFO[SyrcThread；S；FiLeTxnLoge203]-Creatirgn&ulogFile：log.300000001Io回三个窗口都启动后，如下表明Io回R管C:\Windows\system52\cmd.exe-zkSen/er.cmd2017-03-2410；07；34,005[myid；1]-INFO[SessionTrack&r；ZooKe&perSeru&r@35B]-ExpiringsessionCx5afSaaS235CaO3,timeoutofSOOOOnisEHcemdRd2017-03-2410：07：34,0G7[myid：1]-INFO[Proc&GsThiread^sid：1eport：1)：：PrepReq□eatProces3or@4ST]-ProcessedsessionteminationforsessionLd：ax5afSaaS235aO□52C17-C3-2410:07:34,007[myid：1]-INFO[ProcessThread^sid：1eport：-1)：：PrepRequestProceGsor@4B7]-Processedsessionterminationforsessionid：0x5aF8aaS2350G□32017-03-2410:09:43,123[myidl]-INFO[/I27GG1；3SB9；QLiordniCnxHanager$LlgtenerG541]-R^cmiuedconnectionrequest/I：572S22017-03-2410:09:43,132[myidl]-INFO[UorkerReceLuer[niyid=1]；FastLeaderELectioriQSOO]-Notification：1formatuersionl.2i^n.Leader"i,0x20000003d〔n.zxLdi,0x1(nround):LOOKING(n.statei,2(n.sid):0x2[rpeerEpochiLEADING(mystate)2017-03-2410:09:43,201[myidl]-INFO[LearnerHardler-/127001：572B3；LearnerHardler@329]-Followersid：2：irfo：org.apache.zoake@per.seruer.quorum.Quor□[nPeer$QuoruinS&ruer(]5c2b5ba2017-C3-2410:09:43,210[myidl]-INFO[LearnerHardler/12F001；572S3；LearnsrHardl&r@3S4]-SynchronizinguithFolLouersid：2niaxCoiiiniLttedLog=Gx30GOG0GOGniinComniLtt@dLog=Oxl00060001peprLastZxid=0x2O0CaO03d2017-03-2410:09:43,211[myidl]-INFO[LearnerHardler-/127001：572B3；LearnerHardler@45S]-SendingDIFF2017-03-2410:09:43,235[myidl]-INFO[LearnerHardler-/127001：572B3；LearnerHardler@51S]-RpceiuedNELILEftDER-flCkniessag@fron2启动zkclient测试zkserver新开一个cmd窗口，执行如下指令E:cdE:\Apache\zookeeper\zk0\binzkCli.cmd-server:2182^nt：1iyongjiang2017-03-211101S23S68[myid：]-INFO[main：EnuironmentQI00]-Client^nuircriiti^nt：us^r.hoitie：C：MJ^^r^Miyongjiang2017-03-21110：1S：23,S69[myid：]-INFO[main：EnuironmentQI00]-Client^nuircriiti^nt：us^r.dir=E：\fipach^\zook^ep&r\zkG\bin2017-03-21116：18：29:371[myid：]-INFO[main：ZooKepper@43S]■Initiatingclientconnection;conrectString=：21S2seasionTinpout=36060watcher=org.apach口.zookeeppr.ZooKppp?rMain$Mylilatchpr(3277050dcbde1cometo2ooKeeper*2017-03-21116：18：29:995[myid：]-INFO[main-SendThreadi；127.0.0,1：21S2J：CLiertCnxn$SendThiread(31G32]-OpeningsocketconnectiontoseruerIZr.Q.Q.l/IZr.C.e.l：2182WillrotattempttoauthenticateusingS0SL(unknouirerror)2017-03-2Hie；lS：29:997[myid：]-INFO[iiiain-SendThreadC127.Q.Q.1；2182)；CLiertCnsn$SendThir&adeS7G]-Socketconnectionestablishedto127.0.Q.1/I27.Q.Q.1；2182.initiatingsessionJLinesupportisenabled[zk；127.0.0,1；Z182(C0NNECTINGj0]2017-03-2410；10:30,041[niLjid；]-INFO[indin-SendThreadf12T.G.0.1；2182j；ClientCnxn$5endThreadei239]-Sessione^tabli^hitiientcompleteonserver/127.0.0,1；2182.ses^ionid=Oxi1ZdTSOOOO.nwg。tintedtimeout=3g我QWATCHER：：WatchedEu&ntstats：SyncCcnn^ct&dtyp白：N©n电path：null在新开的windows的cmd窗口执行如下命令:create/zkhellocreate/zk/subzkget/zkLs/zk反馈如下:nZxid=0x30000000amtime=FriMar2410：23：^1CST2017p^Kid=cuersion=0dataUersicn=0aclUerGian=0ephemeralOuner=0x0dataLprgth=3numChildrenzQ[zk：：2182[C0NhlEC：TED)4]养t/zkhelloc^Kid=Qxlctiitis=Mar2216：13：03CST2017mZsid=0x100000002mtime:hiedMar2216:13:03昌T201?pZxid=0x1Q0OQ0OO4cuersion=1dataUersicnz0aelU&r5ion=0ephemEralOuner=0x0dataLength=5numChildren=1[zk：127.G.Q.1：2132[C0MhlEDTED)5]1=/zk[subzk][zk：127.C,Q.1：2132rC0NNECTED)G]至此，zookeeper安装完毕4.Kafka安装配置目录配置解压kafka_2.12-.tgz的内容到E:\Apache\kafka目录下，并将文件命名为kafka0,复制该文件夹到相同目录，命名为kafkal,如图kafkaOkafkal在这两个文件夹下创建logs和matrix文件夹，如图(app(E:)卜Apadie卜kafka►kdfkaO►工具(D帮助(H}i防御 *名称 4 修理日朝bin 2017/3/22config 2017/3/22libs 2017/3/22logs 2017/3/23matrix 2017/3/23I site-docs 2017/3/22LICENSE 2017/2/15NOTICE 2017/2/15perties配置修改两个kafka实例的config/perties文件，内容分别如下：kafak0/config/pertiesbroker.id=0port=9092work.threads=2num.io.threads=2socket.send.buffer.bytes=102400socket.receive.buffer.bytes=102400socket.request.max.bytes=104857600log.dirs=./logsnum.partitions=2erval.messages=10000erval.ms=1000log.retention.hours=168log.segment.bytes=536870912erval.mins=10zookeeper.connect=:2181,:2182,:2183zookeeper.connection.timeout.ms=1000000erval.secs=5kafka.metrics.reporters=kafka.metrics.KafkaCSVMetricsReporterkafka.csv.metrics.dir=E:/Apache/kafka/kafka0/matrixkafka.csv.metrics.reporter.enabled=falsekafak1/config/pertiesbroker.id=1port=9093work.threads=2num.io.threads=2socket.send.buffer.bytes=102400socket.receive.buffer.bytes=102400socket.request.max.bytes=104857600log.dirs=./logsnum.partitions=2erval.messages=10000erval.ms=1000log.retention.hours=168log.segment.bytes=536870912erval.mins=10zookeeper.connect=:2181,:2182,:2183zookeeper.connection.timeout.ms=1000000erval.secs=5kafka.metrics.reporters=kafka.metrics.KafkaCSVMetricsReporterkafka.csv.metrics.dir=E:/Apache/kafka/kafka1/matrixkafka.csv.metrics.reporter.enabled=false启动kafkaserver在新开的windows的cmd窗口执行如下命令：E:cdE:\Apache\kafka\kafka1.\bin\windows\kafka-server-start.bat.\config\pertiesordirator.GroupCoordinator)[2017-03-2410:55:12,5SE]IMF。[GroupMetadataManageronHrak&r1]：Remouedaexpiredof千setsin5mi11isecond^.(Kafka.coordinator.GroupMetadataHanager)[2Q17-03-2410:55:12,612]INFOWillnotloadMW+J,mx4j-tools.jarisnotintheclasspath[kafka.utils.Ms1!jLcaderJj[201T-Q3-2410:55:12.682]IMFOCreating/brokers/ids/1[i=itsecufs?false;(kafkautils.ZKCheckedEpheni&ral)[2017-03-2410；55；12.S96]INFOResultofznodecreationi$；OK(kafka.utils.2KCh^ck?dEpheiti&ral)[2Q17-03-24IQ：55：12,701]INFORegisteredbroker1atpath/brokers/ids/1withaddrecsES：ErdPoirt(rdLiyjn96931Li5tenerNanie(PLfirNTEXT).PLAINTEXT;[kafka.utils.欲Utils〕[2Q17-03-2410:55:12.728]IMFOKafkauer^ion：(org.apache.Kmon.utils.flpplnfoParser^[2Q17-03-2410：55：12,741]INFOKafkacommitld：57£d93a8dc0cf421 一叩meh已一mon.utils.AppInfaParser'i[2Q17-03-2410:55:12,F52]INFO[KafkaServer1].startedi；kafka.server.K.afkaSeruer)[2017-03-2410：55：13,337]INFO[R^plicaFetcherManagercnbroker1] f^tcherforpartitions[kafka.aeruer.RpplicaFetcherMarager)[2017-Q3-2410：55；13,QG2]IMF。[ReplicaFetcherManageronbroker1]AddedfetcherforpartitionsList()Ckafka.seruer.ReplicaFetcPierManagerj[2G17-03-241Q：55：13.G93]INFO[icaFetcherManageronbroker1]R«mou&df^tch0rforpartitionst^st-G(kafka.s^ru&r.ReplicaFetcherManagerj在新开的windows的cmd窗口执行如下命令：E:cdE:\Apache\kafka\kafka0.\bin\windows\kafka-server-start.bat.\config\perties的C:\WindDws\systerri22\cnnd.exe.也i门\刈门d!凸犬-gEruerTtqrt.bat"凸1i学席工”[2017-03-2H10；5€；04.317]INFOWillnotloadMX4J.itix4j-tools.jarisnotintheclasspath(kafka.utils.Mx4jLoader$j[2017-03-2410；5E；e4139B]INFOCreating/brokers/idc/Siisitsecure?false)i；kafka.utils.ZKCheckedEphemeralj[2017-03-2410:56:04,420]INFOResultofznodecreationis：OK(kafka.utils.ZKChsckedEphenipral；i[2017-03-2410：5G：O4,i|G1]INFORegisteredbrokerGatpath/brokPr5/ids/Owithaddre^&s：EndPointfrd.liyj.S0S2,Listen«rhlBitieiPLftlNTEXTLPL^INTEXT^fkafka.utils.ZkUtils)[2017-03-2H10；5€；04.512]INFOKafkauersion；G,1D,2.0(tnon.utilf,AppInfoParser)[2017-03-2H10:56:04.530]INF。Kafkaccitttiiitld；5T6d33a5dcGcf421(org.apache.kafmon.utils.flpplnfoParser)[2017-O3-2M10；5G；04?531]INFO[KafkaSeruer0].started[kafka.ceruer.KafkaSeru口「〕[2017-03-21110：56：04,858]INFO[ReplicaFetchcrHanageronbroker0]Rpmouedfetcherforpartitionstest_0(kafka.seruer.ReplicaFctcherManagpr)[2017-03-21110:56:04,8&G]INFOTruncatinglogteat-OtooffsetS.fkafka.log.Log)[2017-03-2M10：5€：O4,915]INFO[R^plicaFcteh^rThrcad-0-1].StartingCkafka.s^ruer.ReplicaFeteherThreadj[2017-03-2H10:56:04.950]INFO[ReplicaFetcherManageronbroker0]AddedfetcherforpartitionsListi；[testG.initOffsetStobrokerBrokerEndPointf1.rd_liyj,3O])i^kafka.seruer.ReplicaFctcherManager)创建主题创建主题，命名为“test2"，replicationfactor=2(因为2个Kafka服务器在运行)。如果集群中所运行的Kafka服务器不止2个，可以相应增加replication-factor，从而提高数据可用性和系统容错性。在新开的windows的cmd窗口执行如下命令：E:cdE:\Apache\kafka\kafka0\bin\windowskafka-topics.bat--create--zookeeper:2181,:2182,:2183--replication-factor2--partitions2--topictest2MicrosoftWindows[版本S.1.7E01]俄校计有（匕）2009MicrosoftCcirpdrmtidri。i呆出产有机利，C：\Users\liyongjiang>E：E：\>cdE：\Apache\kafka\kafkaB\bin\windousE：%Apwctie\k.m"FK£Kafkw6Bin\idinclgi5〉Km+k.m-gpic£,gt--create--zockeeper12T.0.3.1：2.1：Z,1：2183--replication-factor2--partitions2--topictest2Createdtopic"tect；1.E：\Apach»\kafka\kaFkabin\windows>_启动producer在新开的windows的cmd窗口执行如下命令：E:cdE:\Apache\kafka\kafka0\bin\windowskafka-console-producer.bat--broker-list:9092,:9093--topictest2C：\Users\liyongjiang>E：E；\>cdE：\Apach«\kafka\kBfkaG\bin\HindoNSE：X^pach«\kafkaXkafkaQXbin\w±ndows>kafka-consol0-produ»r.bat--brok&r-List127.0.G.1；S092n：9GS3--topicteGt2启动consumer在新开的2