超大规模信息系统核心技术 丁轶群 博士@还是看书课件

1、超大规模信息系统核心技术丁轶群 博士还是看书超大规模信息系统定义超大规模信息系统总体框架超大规模信息系统的一些细节性能建模存储安全消息Agenda超大规模信息系统指的是规模庞大、结构复杂、目标多样、影响因素众多，具有高可靠性、高并发性和高可扩展性的信息系统。其中：规模庞大：主要反映在以下维度1 4 5原代码多：可达到上百万行代码开发人员多：几百上千人，全球多个团队协作开发数据量大2 ：处理的数据达到PB级并发量大：可能会全球上亿人同时访问结构复杂：包括以下几个方面1 4 5组件关系复杂：可能会有遗留系统的组件、第三方组件和自己开发的组件持久化机制复杂：可能有文件、数据库等硬件平台复杂：可能包括

2、嵌入式系统、PC、大型机、手机、传感器等等超大规模信息系统在工业界、政府、军队和科研机构都有广泛的应用3，如一些大型互联网内容提供商、电子商务、文本、视频系统、在线游戏、金融系统、医疗系统、海量科研数据处理，等等。超大规模信息系统定义 1/2超大规模信息系统定义 2/2NameExampleConcurrencyServerLines of codeData sizeSmall system个人网站、公司、政府的门户网站每天几百次访问一台PC机即可几千行MBMedium systemFreecity, MIS, etc几千用户同时访问专门的服务器几万行GBLarge system19楼、天涯、

3、猫扑上万用户同时访问专门的服务器集群几十万行TBVery large systemGoogle, Taobao, Facebook, Youtube, etc. 上亿用户同时访问大型数据中心上百万行PBUltra large systemFutureTBD云计算、互联网上亿行代码EB?超大规模信息系统总体框架 2/2目标：速度 可扩展性 可靠性=自行开发或较大改动=利用现有工具=暂时不做性能专家面对的问题20% 的应用程序存在过性能问题50% 的主管对性能有信心14% 机构有正式的性能管理流程71% 机构性能问题被用户投诉性能建模-问题 1/2A Survey from 150 euro se

4、nior IT Managers ，2006性能建模: 为计算机软件、计算机硬件、系统用户行为建立数学模型,以数学推理方法预测系统在不同负载下的性能性能=f(系统负载，系统资源)性能建模-是什么性能建模-流程性能建模-实例Predict the performance of a Post trade systemTechnology:Struts ServletHibernateJDBC Topology:性能建模-实例性能建模-实例过载控制容量规划定量评估系统组件、系统架构评估新功能、新用户、新平台的性能影响性能建模-应用存储&事务-总体思路 2/2关系型数据库DataFault Toler

5、antDecentralizedScalableHigh PerformanceWeak TransactionProvenStrong transaction supportSometimes an overkillHard to ScaleNOSQLSQL16关系型数据库方面 1/2SSD Base storage optimizationDecrease the small writeDecrease the in-place updateMulti-core based optimizationDecrease the granularity of lockDecrease the c

6、ost of thread switch(membrain)Memory optimizationBetter buffer flush algorithmTotal Order based sync replication云存储-NOSQL方面NOSQL vs. SQL 读写速度19NOSQL成功案例简单的数据模型和操作无需强事务支持高可扩展性高性能海量数据写操作灵活的数据schema高可用性简单管理与维护Twitter tweetsYes, Query by ID and AuthorYesYes, TB YesYesYesFacebook Inbox SearchYesYesYes, T

7、B YesYesYesYesDigg Green BadgeYesYesYesYesYesTaobao Product DetailYesYesYesYesYes20NOSQL的市场需求Market Research Media: NoSQL Market Forecast 2011-201521代表性NOSQL系统工业界Dynamo, AmazonBigtable, GoogleCassandra, ApacheHbase, Apache学术界Scalaris, Zuse Institute BerlinecStore, National University of SingaporeG-S

8、tore, UCSB 22NOSQL存储工作计划Cassandra BasedBigtable style data model can support complex operation while still has high scalabilityIs easy to hacked-in compared with HbaseOptimized for read latencyAtomicity guarantees across multiple keysOptimized for some sub-component, e.g. failure detectorOptimized f

9、or multi-core and flash-storageSecurity, Granular ACL controlMore complex data access model23Cassandra- 横向可扩展性Cassandra-高可用性011/2FEDCBAN=3h(key1)并发读Cassandra-高读写性能 1/2QueryClosest replicaCassandra ClusterReplica AResultReplica BReplica CDigest QueryDigest ResponseDigest ResponseResultClientCassandra

10、- 可动态调整一致性/持久性与延时WriteReadLevelDescriptionLevelDescriptionZEROCross fingersOne1st ResponseOne1st ResponseQUORUMN/2 + 1 ReplicasQUORUMN/2 + 1 ReplicasALLAll ReplicasALL All Replicas一致性模型取决于副本（Replicas）的数量（N）,一般为3 在Cassandra中一般选择Quorum数量的读节点数（R,一般为2）以及Quorum数量的写节点数(W,一般为2)Cassandra-实例 1/3Cassandra-实例

11、2/3Cassandra-实例 3/3恶意代码检测-目标Status update Evaluate existing security products (AppScan, Fortify, Veracode) and find the security issues that they dont coverSurvey on the innovative approaches in research area related to malicious code detectionGoalFind out a security solution for the uncovered secur

12、ity issuesIBM AppScanAn application security testing suite designed to automate vulnerability assessments for web applications, with a black-box approachAppScan EngineSDKUIExtensionsbrowser-side malicious code detection Hybrid JavaScript Analysis34HP FortityAn application security testing suite desi

13、gned to automate vulnerability assessments for software, with both static & dynamic approachesSoftware LifecycleSource Code Analysis (SCA)Program Trace Analysis (PTA)Runtime Time Analysis (RTA)18 Development Languages Java, .NET, C/C+ JavaTomcatWebLogicWebSphereJBossIISLinuxSolarisAIXWindows ServerH

14、P-UX.NETJavaTomcatWeblogicWebsphereJBossIISSolarisMac OS XWindows 2K/2003/XPLinuxWindows Server.NET35Fortify - Static ApproachSource CodeAnalysis EngineControl FlowData FlowNST18 Source codeReport36Fortify - Dynamic ApproachPTA/RTA RulesInterceptCovered Vulnerabilities:SQL InjectionCross-Site Script

15、ing: ReflectiveCross-Site Scripting: PersistentCommand InjectionArbitrary URL RedirectionPath ManipulationHTTP Response SplittingUnhandled ExceptionPrivacy Violation: Social Security NumberPrivacy Violation: Credit Card NumberLog ForgingXpath InjectionLDAP Injection-Buffer overflowCommand injectionX

16、-Site scriptingDirectory listingHTTP response splittingPrivacy violationsProbingSQL injectionInformation leakageUnhandled exceptions37VeracodeA suit of application security testing service designed to automate vulnerability assessments for software, with both static & dynamic & Manual approachesBina

17、ry and/or URLStatic Binary AnalysisDynamic Penetration TestingManual Penetration TestingReportCode security analysis38How Veracode Static binary and Dynamic approach worksProgram BinaryLoaderdisassemble the binaryApplication ModelerCreate a model of the application including data flow, control flow,

18、 and etc.Model AnalyzerPerform analysis by scanning the modelReporterGenerate security metrics & correlate security flaws with original source codeSecurity MetricsDetailed Flaw ListURLNTOSpider Engineenumerates the web siteAttack phasesubmits specially crafted requests to trigger potentially malicio

19、us application behaviorReportStatic binary analysisDynamic analysis39Survey results of Research malware detection40How Do We Differ?FVFVAVAVOurTeamDynamicStaticPenetration TestSoftware WeaknessDetection ApproachF=FortifyV=VeracodeA=AppScan41Dynamic ComponentMalicious Code Detection Platform42QEMUTEM

20、UMalware AnalysisVulnerabilitydetection, diagnosis and defenseProtocol Reverse EngineeringPanoramaRevonoHookFinderBitScopePolyglotDispatcherstingAPEGMineSweeperAnubisGFI SandBoxNorman SandBoxJoe SandBoxStatic Component42消息中间件-现状与目标43FunctionalityNon-FunctionalHRZARich FunctionalityScalable, Reliable

21、 Based on Erlang OTP AMQPFast PersistenceJournal solutions AIOHigh PerformanceBrokerlessAsync, Non-blockingZero copy, Buffer 2004(ASL)2008(MPL)2009(ASL&LGPL)2010(LGPL)为什么要开发自己的消息中间件1. VLIS Messaging TargetHigh performance, high scalability, high reliability for financial transaction.Light-weight, lo

22、w-cost, easy-to-use.Commercialization.2. Why not choose an open-source product?No products can satisfy all of the above requirements.Cant follow technology innovation, optimization necessarily.License barrier.3. Our Strategy:Following design philosophy.Light-weight, plugable.44目标High PerformanceI/O

23、& Concurrency Model: Asynchronous, Non-BlockingMemory Management: Different sizes processing, Zero-copy, Mem_pool Cache Locality: Time & SpaceHigh Scalability Load balancing: Static & Dynamic IPC via messages: Take advantage of multi-core Lock-free: CAS(Compare And Swap)3. High Reliability High avai

24、lability: Master & Slave, Failure detection(Heart-beating), Recovery strategy(backoff, Idempotency) Persistence: High Performance Journal solution45Monitoring-基于现有工具的平台层监控Schedule(周晓震+李伟+王耿亮+1RA)9月:选型10月:安装使用，总结经验，尝试开发插件11-12月:开发用于中间件的插件，并与我们的平台集成1月:测试46缓存-基于现有工具的分布式缓存开源软件名称功能简介Memcached分布式缓存方案，共享缓存

25、方式Ehcache支持分布式缓存设计，支持RMI, JGroup, JMS进行通讯，支持replication和invalidation同步信息JBossCache支持分布式缓存配置，支持严格事务处理SwarmCache轻量级分布式缓存软件JCS支持对象存储的分布式缓存软件dbcached基于memcached协议的一款分布式缓存软件，支持事务功能Hbase基于Big Table的Key-Value文件存储系统，支持分布式存储OSCache大而全的缓存软件，通过不同的配置可以实现集群中的分布式缓存Scalaris比较新的开源分布式缓存软件，Key-Value类型，支持事务处理MongoDB分布

26、式文件存储系统软件47Micro Container-微容器的职责业务层服务对象的生命周期管理向业务层对象提供由其他中间件提供的企业服务业务层应用部署和管理暴露系统服务、业务层服务管理员接口微容器自身运行所需结构：类加载器层次结构、异常处理Application LevelProgramming TechniquesSystem LevelProgramming TechniquesArchitecture DesignDomain KnowledgeStrutsSpringHibernateSocket programmingEclipseSVN/CVSConcurrency modelCl

27、ass loader hierarchyMemory managementMulti-thread programmingCachingConsensus AlgorithmsQueuing network modelPetri NetBayesian Network ModelWavelet TransformationEXP合作机会合作计划成立兴趣小组学习反馈加入团队姓名、学号、电话、参与小组尹可挺老师报名流程：消息中间件兴趣小组学习计划和方法:学习网络协议: TCP/IP协议详解卷1（了解基本的网络协议方面的内容）学习基本的socket网络编程 UNIX网络编程第三版卷1 (重点章节:

28、2，3，4，5，6，8，16，30)通过阅读开源库来掌握网络编程的基本IO模型 对于懂java的同学: 了解java IO和NIO中socket的一些编程方法，并且能够通过阅读源代码了解网络开源库mina, netty，grizzly之一的基本工作原理。 对于懂C/C+的同学: 通过阅读源代码了解libevent的基本工作原理。阅读hornetQ的快速指南和用户手册。/hornetq/chinesedocs.html阅读我的博客上对zeroMQ的分析并且结合zeromq源代码理清思路。/kaka11Output（以下任务均不必掌握细节，目前这个阶段就是要掌握基础的原理）:UNIX网络编程重点章节的读书笔记。对应任务A、B，2周写下libevent，mina, netty，grizzly其中一个的源代码分析来介绍基本工作原理。对应任务C，2周hornetQ用户手册的读书笔记。对应任务D，2周学习计划和方法:了解NoSQL相关概念:/wiki/NoSQL阅读相关经典论文前