让HTAP更简单：谬误与陷阱

HTAPMadeSimple

FallaciesandPitfalls

RossiSun

AboutMe

●RossiSun(孙若曦)

●Techlead,ComputeArch.&Engineteam@PingCAP

●Was

○Techlead,SQLonHadoopteam@Transwarp

○Techlead,GPUArchInfrateam@NVIDIA

●Over10-yearexperienceoninfra

○System-levelsoftware

○Databasekernel&big-data

○GPUtechniques

HTAP

●HTAP:HybridTransactional/AnalyticalProcessing

●CoinedbyGartnerin2014

○ReatimeAnalytics

○Analyticsseetheexactoperationaldata

○Simplifydatainfrastructure(nodatamovement)

●Goingmainstream

○20%databasebuyerscitedHTAPcapabilities

○40%markedHTAPasTop3factor

TiDBArchitecture1.0

Pitfall:Non-ScalableAP

●Auto-sharding

●Distributedtransaction

●TPscaleswell

●AP:wecandojoinandaggregation,onlythatonasinglenode

SELECTCOUNT(*)FROM

t0JOINt1

ONt0.c1=t1.c1

WHERE

t0.c2

t1.c2

=xxxand

=yyy

IntroducingTiSpark(Since2.0)

IntroducingTiSpark(Cont.)

id

0962

7658

3589

5523

name

Jane

John

Jim

Susan

age

30

45

20

52

Fallacy:OneSizeFitsAll

●The“Hybrid”…what?

●RowstoreperformspoorforAP

id

name

age

0962

Jane

30

7658

John

45

3589

Jim

20

5523

Susan

52

OLAP

OLTP

Fallacy:OneSizeFitsAll(Cont.)

●TPisseverelyinterfered

IntroducingColumnarStore(POC)

Pitfall:UnboundedStaleness

Pitfall:Complexity

EmbracingRaft(Since4.0)

IntroducingMPP(Since5.0)

IntroducingMPP(Cont.)

Fallacy:Consistency&Freshness

●UseMVCC+learnerread(raft)toguaranteedataconsistencyandfreshness

●Justlikeserializableisolationlevel

○Ideal,butatthecostofperformance

Timestamp:11balance=?

RaftLeader

Fallacy:Consistency&Freshness(Cont.)

MVCC

...

TS:10

$100

RaftLearner

4

2

MVCC

Timestamp:11balance=$200

RaftLeader

...

TS:10

$100

TS:11

$200

TS:12

$150

Fallacy:Consistency&Freshness(Cont.)

RaftLearner

4

4

MVCC

.

..

TS:10

$100

TS:11

?

Timestamp:11

Staleness:-5

balance=?

RaftLearner

4

StaleRead(Since6.6)

●Boundedstaleness

●Dataintegrity

●Gets30%moreQPSinourinternalPOC

X

balance=$100

RaftLeader

2

Fallacy:SharedNothing

Fallacy:SharedNothing(Cont.)

DisaggregatedCompute/Storage

●AvailableonTiDBCloudServerlessTier

DisaggregatedCompute/Storage(Cont.)

●Comingin7.0

Fallacy:HTAP=TP+AP

●ThesubtleboundarybetweenTPandAP

Fallacy:HTAP=TP+AP

●ThesubtleboundarybetweenTPandAP

Windowfunctionon

16mrows

80kqps

5mslatency

Heavyaggregationon

6brows

50qps

10slatency

OneOptimizorToRuleThemAll(Since6.5)

●Casestudy:Topreposongithuboveraspecifiedperiod

320mrows

5brows

<10rows

OneOptimizorToRuleThemAll(Cont.)

<10rows

320mrows

5brows

OneOptimizorToRuleThemAll(Cont.)

<10rows

320mrows

1mrows

OneOptimizorToRuleThemAll(Cont.)

320mrows

14mrows

<10rows

ComputeEngineEvolvement

●Casestudy:CPUunderutilization

ComputeEngineEvolvement(Cont.)

●Thedevilinstd::thread:

○lll_lock

○stack_cache

○deallocate_stack

○munmap

○TLBshootdown

○IPI

●Mitigation:Dynamicthreadpool(since5.4)

ComputeEngineEvolvement(Cont.)

●Moderateconcurrencybringsnewchallengestocomputeengine:

○Out-Of-Memory

○Out-Of-Thread(exceedsthemaxmmapulimit)

○Lackoffine-grainedschedulingandtaskprioritization

●Mitigation:

○Memorytracker(since6.1)andspilldisk(comingin7.0)

○Min-TSOscheduling(since6.0)

○Push-modelandpipelineexecution(comingin7.0)

Fallacy:TheCostofHTAP

HTAPServerless&Elasticity

●CominginTiDBCloudServerlessTier

●Pay-As-You-Go,costmetricsre-designed

○Datastored

○Read/write/computation

●Extremeelasticity

○Poolinginstancestoreducecoldstarttimetosecond-level

○Scale-intozero