TiDB架构设计指南

1、TiDB架构设计指南TiDB Design And ArchitectureWhy we need a new databaseThe goal of TiDBDesign & ArchitectureStorage LayerSchedulerSQL LayerSpark integrationTiDB on Kubernetes From scratch Whats wrong with the existing DBs?RDBMSNoSQL & Middleware NewSQL:F1 & Spanner1970s20102015PresentMySQLPostgreSQL Oracle

2、DB2.Redis HBase Cassandra MongoDB.Google Spanner Google F1 TiDBRDBMSNoSQLNewSQLScalabilityHigh AvailabilityACID TransactionSQLA Distributed, Consistent, Scalable, SQL Database that supports the best features of both traditional RDBMS and NoSQLOpen source, of courseData storageData distributionData r

3、eplicationAuto balanceACID TransactionSQL at scaleApplicationsMySQL Drivers(e.g. JDBC)TiDBTiKVMySQL ProtocolRPCSQL LayerStorageLayerGood start! RocksDB is fast and stable.Atomic batch writeSnapshotHowever Its a locally embedded KV store.Cant tolerate machine failuresScalability depends on the capaci

4、ty of the disk Fault ToleranceUse Raft to replicate dataKey features of RaftStrong leader: leader does most of the work, issue all log updatesLeader electionMembership changesImplementation:Ported from etcdReplicas are distributed across machines/racks/data-centers Fault ToleranceMachine 1Machine 2M

5、achine 3RocksDBRocksDBRocksDBRaftRaft ScalabilityWhat if we SPLIT data into many regions?We got many Raft groups.Region = Contiguous KeysHash partitioning or Range partitioning?Redis: Hash partitioningHBase: Range partitioningRange Scan:Select * from t where c 10 and c a - dRegion 2 - e - hRegion n

6、- w - zData is stored/replicated/scheduled in regions(-, +) Sorted MapLogical KeySpace Meta: Start_key, end_key)Thats simpleLogical splitJust Split & MoveSplit safely using RaftRegion 1Region 1Region 2Region1Region3Region 1Region 2Region 1*Region 2Region 2Region 3Region 3Node ANode BNode CNode DRegi

7、on1Region3Region1Region 2Region 1*Region 2Region 2Region 3Region 3Node BNode ANew NodeENode CNode DRegion1Region3Region1*Region 2Region 2Region 2Region 3Region1Region 3Node ANode BNode CNode DNew NodeERegion 1Region1Region3Region1*Region 2Region 2Region 2Region 3Region1Region 3Node ANode BNode CNode

8、 DNew NodeEClientStore 1Region 1Region 3Region 5Region 4Store 3Region 3Region 5Region 2Store 2Region 1Region 3Region 2Region 4Store 4Region 1Region 5Region 2Region 4RPCRPCRPCRPCTiKV node 1TiKV node 2TiKV node 3TiKV node 4Placement DriverPD 1PD 2PD 3Raft GroupMVCCData layoutkey1_version2 - valuekey1_

9、version1 - valuekey2_version3 - valueLock-free snapshot readsTransactionInspired by Google PercolatorAlmost decentralized 2-phasecommitHighly layeredRaft for consistency and scalabilityNo distributed file systemFor better performance and lower latencyTransactionMVCCRaftKVLocal KV Storage (RocksDB)Pr

10、ovide the Gods view of the entire clusterStore the metadataClients have cache of placement information.Maintain the replication constraint3 replicas, by defaultData movement for balancing the workloadIts a cluster too, of course.Thanks to Raft.Placemen tDriver Placement DriverPlacement DriverRaftRaf

11、tRaftNode 1Region ARegion BNode 2PDScheduling StrategyCluster InfoAdminHeartBeat with InfoSchedulin g CommandRegion CConfi gMovementReplica number in a raft groupReplica geo distributionRead/Write workloadLeaders and followersTables and TiKV instancesOther customized scheduling strategySQL is simple

12、 and very productiveWe want to write code like this:SELECT COUNT(*) FROM userWHERE age 20 and age 10Can we push down filters?select count(*) from personwhere age 20 and age 20 and age 20 and age 20 and age 30TiDB knows that Region 1 / 2 / 5stores the data of person table.We just build a protocol lay

13、er that is compatible with MySQL. Then wehave all the MySQL drivers.All the toolsAll the ORMsAll the applicationsThats what TiDB does.KV APICoprocessorTxn, TransactionMVCCRawKV, Raft KVRocksDBPlacement DriverMySQL clientsLoad Balancer (Optional)MySQL ProtocolTiDB SQL LayerKV APIDistSQL APITiDB Serve

14、r (Stateless)MySQL ProtocolTiDB SQL LayerKV APIDistSQL APITiDB Server (Stateless)Pluggable Storage Engine (e.g. TiKV)TiSpark = Spark SQL on TiKVSparkSQL directly on top of a distributed Database StorageHybrid Transactional/Analytical Processing(HTAP) rocksProvide strong OLAP capacity together with T

15、iDBSpark ecosystemTiDBTiDBWorkerSpark DriverMeta dataTiKVTiKVTiKVApplicationSyncerData locationJobTiSparkDistSQL APITiKVTiDBTSO/Data locationWorkerWorkerSpark ClusterTiDB.TiDB Cluster.TiKV Cluster (Storage).DistSQL APIPDPDPDPD ClusterTiKVTiKVTiDBTiKV Connector is better than JDBC connectorIndex supp

16、ortComplex Calculation PushdownCBOPick up the right Access PathJoin ReorderPriority & Isolation LevelAPI ServerSchedulerKubernetes CoreController ManagerTiDB OperatorDeploymentTiDB Cluster ControllerPD ControllerTiKV ControllerTiDB ControllerGC ControllerVolume ManagerDaemonSetTiDB Scheduler: Kube Scheduler + Scheduler Extender DaemonSetUsersUsersAdminTiDB Cloud Manager RESTFul Interface Extern