版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领
文档简介
BIGDATAEVERYMINUTE…1,388
cabs2,777privatecarsDidirideshailed:EVERYMINUTE…395,833PeopleloginToWeChat194,444peoplearevideooraudiochattingEVERYMINUTE…625,000YoukuTudouvideosbeingwatchedEVERYMINUTE…64,814postsandrepostsonWeiboSEARCH4,166,667
searchqueriesEVERYMINUTE…774
peoplebuysomethingonAlibaba’smarketplacesUS$1,133,942spentonAlibaba1Definition2Characteristic3NoSQL4RDBMS5MapReduceCONTENTS6Applications1Definition1DefinitionBIGDATAvolumeofdataimportantdataonaday-to-daybasisforbetterdecisions2Characteristic2CharacteristicVolumeThequantityofgeneratedandstoreddata.VarietyThetypeandnatureofthedata.Thequalityofcaptureddatacanvarygreatly,affectingaccurateanalysis.VelocityInthiscontext,thespeedatwhichthedataisgeneratedandprocessedtomeetthedemandsandchallengesthatlieinthepathofgrowthanddevelopment.VariabilityInconsistencyofthedatasetcanhamperprocessestohandleandmanageit.Veracity3NoSQL3NoSQLNoSQLreferstodocument-orienteddatabasesSQLdoesn’tscalewellhorizontally.Itisschemaless.Butnotformless(JSONformat).JSON:datainterchangeformatMongoDatabaseCouchDatabase3NoSQLBasicAvailabilityspreaddataacrossmanystoragesystemswithahighdegreeofreplication.SoftStateEventualConsistencyBaseModeldataconsistencyisthedeveloper'sproblemandshouldnotbehandledbythedatabase.atsomepointinthefuture,datawillconvergetoaconsistentstate.Noguaranteesaremade“when”.3NoSQL{field1:value1,field2:value2…fieldN:valueN}varmydoc={_id:ObjectId("5099803df3f4948bd2f98391"),name:{first:"Alan",last:"Turing"},birth:newDate('Jun23,1912'),death:newDate('Jun07,1954'),contribs:["Turingmachine","Turingtest",…],views:NumberLong(1250000)}JSONStructure3NoSQLRDBMSvsNoSQLXszcRowDB:001:10,Smith,Joe,40000;002:12,Jones,Mary,50000;003:11,Johnson,Cathy,44000;004:22,Jones,Bob,55000;index:001:40000;002:50000;003:44000;004:55000;ColumnDB:10:001,12:002,11:003,22:004;Smith:001,Jones:002,Johnson:003,Jones:004;Joe:001,Mary:002,Cathy:003,Bob:004;40000:001,50000…;Smith:001,Jones:002,004,Johnson:003;…3NoSQLBenefitsColumn-orientedorganizationsaremoreefficientwhenanaggregateneedstobecomputedovermanyrowsbutonlyforanotablysmallersubsetofallcolumnsofdata,becausereadingthatsmallersubsetofdatacanbefasterthanreadingalldata.Column-orientedorganizationsaremoreefficientwhennewvaluesofacolumnaresuppliedforallrowsatonce,becausethatcolumndatacanbewrittenefficientlyandreplaceoldcolumndatawithouttouchinganyothercolumnsfortherows.Row-orientedorganizationsaremoreefficientwhenmanycolumnsofasinglerowarerequiredatthesametime,andwhenrow-sizeisrelativelysmall,astheentirerowcanberetrievedwithasinglediskseek.Row-orientedorganizationsaremoreefficientwhenwritinganewrowifallofthecolumndataissuppliedatthesametime,astheentirerowcanbewrittenwithasinglediskseek.3NoSQLSQLvsNonSQLAgoodcompromiseistodesignyoursystemwith3logicalDBs1.NormalSQLDBusedbyyouradminapplicationtocreatecontent.2.No-SQLDBforfront-end/public/high-volumeapplicaitonusedbythepublicinternet.3.ThelastDBisforanalyticalreportingsystemusingcubesandallthatgoodstuff.ThendataflowsfromtheAdminDBtotheclientNo-SQLDBwhensomeone"Publishes"apieceofcontent,theclient(NoSQL)dbprovidesveryfastreadaccessandrecordsuserinteractionswiththecontent.ThenyouhaveascheduledjobthatpullsthedatafromtheclientDBintothereportingsystem.SinceAdmin,client,andreportingareoftenseparateapps,eachapplicationteamcanworkwithdataintheformatthatbestservestheapplicationandthetransitionfromonesystemtotheotherishandledintheservicelayers.4RDBMS4RDBMSfixed-schema,row-orienteddatabaseswithACIDpropertiesandasophisticatedSQLqueryengineTheemphasisisonstrongconsistency,referentialintegrity,abstractionfromthephysicallayer,andcomplexqueriesthroughtheSQLlanguage.easilycreatesecondaryindexes,performcomplexinnerandouterjoins,count,sum,sort,group,andpageyourdataacrossanumberoftables,rows,andcolumns.5MapReduceDividingandconqueringHighlyfaulttolerantEverydatablockreplicatedon3nodesDifficulttoimplement5MapReduce5ComparisonRDBMSMapReduceDatasizeGBPBAccessInteractiveandBatchBatchUpdatesRead/WritemanytimesWriteonce,ReadmanytimesStructureStaticSchemaDynamicSchemeIntegratedHigh(ACID)LowScalingNolinerLinerDBARatio1:401:30005HowdoesMapReduceworkMapReduceuseskey/valuepairs.(Traditionallyusingrowsandcolumns) --Mapalltheintermediatevaluesforagivenoutputkeyarecombinedtogetherintoalist. --ReduceThereducefunctionthencombinestheintermediatevaluesintooneormorefinalvaluesforthesamekey.--ReduceTwosteps:MapandReduce6Application6GovernmentTheuseandadoptionofbigdatawithingovernmentalprocessesisbeneficialandallowsefficienciesintermsofcost,productivity,andinnovation,butdoesnotcomewithoutitsflaws.Dataanalysisoftenrequiresmultiplepartsofgovernment(centralandlocal)toworkincollaborationandcreatenewandinnovativeprocessestodeliverthedesiredoutcome.Belowarethethought[bywhom?]leadingexampleswithinthegovernmentalbigdataspace.6HealthcareBigdataanalyticshashelpedhealthcareimprovebyprovidingpersonalizedmedicineandprescriptiveanalytics,clinicalriskinterventionandpredictiveanalytics,wasteandcarevariabilityreduction,automatedexternalandinternalreportingofpatientdata,standardizedmedicaltermsandpatientregistriesandfragmentedpointsolutions.6EducationAMcKinseyGlobalInstitutestudyfoundashortageof1.5millionhighlytraineddataprofessionalsandmanagersandanumberofuniversitiesincludingUniversityofTennesseeandUCBerkeley,havecreatedmastersprogramstomeetthisdemand.Privatebootcampshavealsodevelopedprogramstomeetthatdemand,includingfreeprogramslikeTheDataIncubatororpaidprogramslikeGeneralAssembly.6InternetofThingsBigDataandtheIoTworkinconjunction.Fromamediaperspective,dataisthekeyderivativeofdeviceinter-connectivityandallowsaccuratetargeting.TheInternetofThings,withthehelpofbigdata,thereforetransformsthemediaindustry,companiesandevengovernmen
温馨提示
- 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
- 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
- 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
- 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
- 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
- 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
- 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
最新文档
- 初中历史事件时间表详述
- 人教版量词搭配难题破解
- 离子反应的平衡常数
- 新版苏教版小数除法教学设计解析
- 人教版高中文学常识提炼
- 初中生物苏教版复习资料模板
- 初中地理人教版目录学习方法分享
- 初中苏教版语文经典文言文解析
- 北师大英语必修一的核心知识点
- 人教版地图上的方向与位置
- 退伍军人职业规划课件
- 新概念英语第一册课文(全英文)
- 焊工安全培训(复审)课件
- 小学主题班会之团结友爱班队会课件
- 常用抢救药品作用及不良反应课件
- 人教PEP英语五年级上册unit-4-What-can-you-do单元复习课(赛课)课件
- 2023年济南出版有限责任公司招聘笔试模拟试题及答案解析
- 49种美丽的兰花课件
- 鲁科版英语小学三年级上册Unit-2-Lesson-1《This-is-WangHong》课件
- 妊娠期补钙补铁课件
- 中班体育游戏教案大全200篇
评论
0/150
提交评论