ArchSummit全球架构师峰会-《华为Git实践：工作模式创新及多中心分布式架构 》－蒋鑫

工作模式创新，及多中心分布式架构蒋鑫‣Gitcontributor(gitclean-i,...)‣Gitl10ncoordinator(/git-l10n/git-po)‣《Git权lub》,2011(/gotgit/gotgit)‣2005—2015L自雇佣L前一半时间vpSVNLe一半时间h纠正这O错误‣2015.12加入华PiSourceg队๏“买买买?还是自研?”跳出同质竞争的红海跳出同质竞争的红海驶入模式创新的蓝海GiEHFbAoHeCedE:eH:oleoAenDoFCce阿里腾讯GiEHFbAoHeCedE:eH:oleoAenDoFCce阿里腾讯MicrosoftVSTS度OMEGA.Distributed;freedom;commitanywhere,anytime.Geeky...Fork+PullRequestSocialSocial.Syncofforkrepos;wasteofdisk.spacesMulti-repomanagement,gitsubmodule?..ReviewinGooglestyleMulti-reposolution:repo...Projectbyproject,notaplatformCentralized,lackoffreedomJustforreview,andtheUI•Git工作流创新•多中心Git架构clone分布式工作流的困扰1：服务器端的存储压力cloneFreshclone,andhardlinksAfternewpushandGCfork origin originRepo100分布式工作流的困扰2：客户端操作的复杂性fork origin originRepo100upstreaupstream 本地gitremoteaddupstreamURLgitfetchupstreamgitrebaseupstream/mastergitpush-foriginHEADReverseMergeRequestupstreaupstream 本地owabout 本地forkFork1RepoFork1forkFork2RepoFork2......forkForkFork100•slowdownclone/fetch•loosereferences,highIO•chaosofvisibilityofallrefs•Multiplereposinoneproject,why?‣GC,clonearequiteslowforbigrepos;auth;microservices•gitsubmodule：适用范围：仓库少，松鹅合‣Recurisivesubmodules:描述复杂的目录嵌套‣Updateofgitlinks.迟更新和超前更新问题‣Conflictofgitlinks.支持几万研发，简直是灾难仓仓nosubmodulesOne-stopMulti-EndpointsGitAccessAPI#2merge-requestAPI#4GETsuper-mr/:idAPI#3super-mrAPI#1GET/ssh-infogitpush--receive-pack=omegaorigin\HEAD:refs/for/target-branch/local-branchgitlab-shell(omega:能读即能写)•••‣CCe3Ee-RMRbyAFD:‣DC3fEmode-RMROMEGA模式操作示例gitpushoriginHEAD:refs/for/master/my/topic1#moregitcommit...gitpushoriginHEAD:refs/for/master/my/topic1gitpushoriginHEAD:refs/for/master/my/topic2cloneclone集中式工作流典型分布式工作流派派生克隆本i分yw交v送mtMR创rfM游d步xXMR说明分y比较c起MR删除分yritOMEGA服务名--receive-pack="gerritreceive-pack[options]"--receive-pack="omega[options]"代码检视Change(percommit)MergeRequest(perfeature)特殊refspecHEAD:refs/for/masterHEAD:refs/drafts/masterHEAD:refs/for/master/local/branchHEAD:refs/drafts/master/local/branch提交和检视任务的关联•githook:commit-msg•在提交说明中嵌入唯一的标识•Change-Id:I......•Noclientsidehooks•无须在提交说明中嵌入特殊标识•同一个用户/本地分支对应同一个MR多仓库管理模式Manifest客户端工具repo•OnlyforLinux•Onlysupportmultiplerepos•Clonefrombundlegit-mm•支持Linux、Windows等•既支持多仓模式，也支持单仓操作•支持git-lfs•支持bundle*•支持batchquery*MMXML<?xmlversion="1.0"encoding="UTF-8"?><manifest><remotename="iSource".../><defaultrevision="master".../><projectpath=...MMXML<?xmlversion="1.0"encoding="UTF-8"?><manifest><remotename="iSource".../><defaultrevision="master".../><projectpath=...name=...groups=.../><projectpath=...name=...groups=.../>.....</manifest>•Git-MM:GitMulti-ModuleL参考Androidrepo方案L是适配OMEGA的repoL用Golangsc。•w供统一的Git环境设置。参考:Autodesk/enterprise-config-for-git•d时w供kRfaR解Y方案。XML‣gitconfig--globalalias.review“git-mm--single”XML•Bundle和batchrequest‣gitmminit-uURL‣gitmmsync‣gitmmupload‣gitreview•Git工作流创新•多中心Git架构gitfetch-pack/gitsend-packCGIgitfetch-pack/gitsend-packCGIgitupload-pack/gitreceive-packGit仓库存储客户端传输层服务端GitoverSSHGitlabWebUIGitoverHTTPNginxOpenSSH.ssh/authorized_keysgitlab-shellEgitlab-workhorseauthreqauthreqNginxOpenSSH.ssh/authorized_keysgitlab-shellEgitlab-workhorseauthreqauthreqEDatabasedisCPUgitupload-pack/gitreceive-packgitupload-pack/gitreceive-pack高IO容量Unicornsidekiqgitlab-railssidekiqAPIRuggedlibgit2Git仓库存储GitGitGerritMySQLOpenSSHWebWebouteouteGitGitGitSpokes(DGit),2016BigtableADBKGit+RPCALEMGit+RPCAMCQGit+RPCEGFHEGFHGit+RPCGit+GitGitGerritMySQLOpenSSHWebWebouteouteGitGitGitSpokes(DGit),2016BigtableADBKGit+RPCALEMGit+RPCAMCQGit+RPCEGFHEGFHGit+RPCGit+RPCJLKMJLKMGit+RPCGit+RPCGitGitGitGCDRBDhealing&re-balancejGoogle分布式存储方案GitHub本地存储分片方案OpenSSHMySQLMySQLouteSSHTunnelGGFSGit+RPCAACCBBDDGit+RPCAACCBBDDUnicorngitlab-railsclientUnicorngitlab-railsclientOpenSSH(AuthUnicorngitlab-railsclientUnicorngitlab-railsclientOpenSSH(AuthbyMySQL)gitlab-shellOpenSSH(AuthbyMySQL)gitlab-shellSSHTunnelGit+RPCGit+RPCNAS2NAS3Git+RPCNAS1DatabaseGitoverSSHGitlabWebUIGitoverHTTPLB(LVS-DR)NNginxgitlab-workhorseNginxgitlab-workhorseSSHTunnelGSAND(routingmicroservice)dis•Git靠近存储，无NFS•应用层三副本复制NginxRemoteTunnelNginxgitlab-workhorseOpenSSHgitlab-shellOpenSSHgitlab-NginxRemoteTunnelNginxgitlab-workhorseOpenSSHgitlab-shellOpenSSHgitlab-shellNginxGSAND-MDCMsgQueueAmirrorAmastergitlab-workhorseLocalTunnelGitSyncGitlab社区版本的架构演进TimetoLeavetheCloud》thecloud》infrastructurecompany”stingsolutionforaproblem”GitoverSSHGitlabWebUIGitoverHTTPNginxgitlab-workhorseOpenSSHgitlab-shellauth(shard) auth(shard)gitalyhandlergitaly-upload-packgitalyNginxgitlab-workhorseOpenSSHgitlab-shellauth(shard) auth(shard)gitalyhandlergitaly-upload-packgitaly-receive-packGit