语音编码关键技术及应用PPT课件.ppt

中 低速率语音编码关键技术及应用 电子工程系崔慧娟10 4 15 各种压缩编码算法重建语音实例 原始 8kHz采样 16bit 样点 128kb s 国际标准ITU TG 72816kb sITU TG 7298kb sITU TG 723 15 3kb sCVSD16kb s4 75kbpsAMR4 8kbpsEVRC 我们的声码器MPD ACELP4kb sSELP2 4kb sSELP1 2kb sSELP0 8kb sSELP0 6kb sSELP0 3kb s B 2 3 内容 音频编码性能评价F目前水平F音频压缩依据F人耳听觉特性F现有标准F语音生成模型及参数编码FAdaptivePredictiveCoding APCFAnalysis by SynthesisCodingofSpeechFPerceptuallyWeightedFilterFTheDOD4 8kb sStandard CELPF 音频编码性能评价 1 编码速率 kb s 信号带宽300 3400HZ50 7000HZ20 15000Hz10 20000HZ采样速率8KHZ16KHZ32KHZ44 1 48KHZ 编码位数R b ps 总速率I kb s 可懂度 自然度 透明度影响重建质量 存储容量 传输带宽 4 音频编码性能评价 2 2 重建语音质量客观评价 信噪比 15dB以上较好 20dB以上相当好 分段信噪比PESQ Perceptualevaluationofspeechquality 5 6 PESQ Perceptualevaluationofspeechquality ITU TRecommendationP 862Anobjectivemethodforend to endspeechqualityassessmentofnarrow bandtelephonenetworksandspeechcodecsTheclosenessofthefitbetweenPESQandthesubjectivescoresmaybemeasuredbycalculatingthecorrelationcoefficient Normallythisisperformedonconditionaveragedscores aftermappingtheobjectivetothesubjectivescores 音频编码性能评价 3 PESQ ThecorrelationcoefficientiscalculatedwithPearson sformula Inthisformula xiistheconditionMOSforconditioni andistheaverageovertheconditionMOSvalues xi yiisthemappedcondition averagedPESQscoreforconditioni andistheaverageoverthepredictedconditionMOSvaluesyi For22knownITUbenchmarkexperiments theaveragecorrelationwas0 935 Foranagreedsetofeightexperimentsusedinthefinalvalidation experimentsthatwereunknownduringthedevelopmentofPESQ theaveragecorrelationwasalso0 935 音频编码性能评价 4 7 2 重建语音质量主观评价 1MOS分 MeanOpinionScore 5 1分 ExcellentGoodFairPoorBad 音频编码性能评价 5 8 2 重建语音质量主观评价 2DRT DiagnosticRhymeTest 正确 错误 总 100 例如 为 wei 费 fei 95 以上优秀85 94 良好75 84 中等65 74 差65 以下不能接受判断可接受度测试DAT DiagnosticAcceptabilityTest 多维因素测试调制噪声参考单位MNRU ModulatedNoiseReferenceUnit 量化失真单位QDU QuantizationDistortionUnit 一次PCM编解码 音频编码性能评价 6 9 3 编解码延时 ms 公众网 25ms 点对点 广播 存储回声控制或回声抵消正常通话秩序与重建质量关系4 算法复杂度硬件 成本浮点 定点MIPs RAM ROM5 其他抗随机误码和突发误码能力抗丢包和丢帧能力对不同信号编码能力级联或转接能力B 音频编码性能评价 7 10 11 目前水平 目前发展水平 宽带音频 宽带语音高质量 2b ps下一步 1b ps电话语音高质量 1b ps下一步 0 5b ps B 音频压缩依据 时域样点之间相关 短时 长时 F 频域谱的非平坦性 谱包络 谱离散 统计特性 语音信号的统计特性 F熵编码 1 冗余度 2 人耳听觉特性 人耳对不同频段声音的敏感程度不同 通常对低频比对高频更敏感 人耳对语音信号的相位不敏感 人耳掩蔽效应MaskingEffect 对人耳听不到或感知极不灵敏的声音分量都不妨视为冗余 可利用听觉心理特性 感觉加权 量化 去除多余分量 后滤波 B 12 语音信号是非平稳随机过程时变性短时平稳性 10 20ms 分帧处理 语音信号的特点 B 13 语音信号的统计特性 短时平稳段分类无话 信息最少 清音 信息较少 浊音 信息较多 起始 信息最多 B 14 人耳听觉特性 1 正常人的听域和听阈正常人可以听见的频率范围为0 016 16kHz 强度范围0 120dBSPL 声压级 这里的基准声压 0dBSPL 是或 自由场听阈是指人进入声场以后能听到的最低声压级 纯音听阈是一个与频率有关的量 1000Hz时 约为4dB左右 而在40Hz时 上升为50dB左右 在15kHz时 上升为24dB左右 感觉阈代表可以容忍的最高声压 在声压高到一定的程度时 耳朵会出现不适的感觉 或者具有痒 压迫及痛感 对正常人而言一般取120dB为不适阈 140dB为痛阈 而且认为其与频率无关 15 16 人耳听觉特性 2 音调 Pitch 音调是在分辨声音频率高低时 用于描述这种感受的一种特征 对于频率低的声音 听起来感觉它的音调 低 反之 听起来感觉它的音调 高 但是音调与频率并不是成正比的关系 它还与声音的强度和波形有关 音调用美 Mel 标度高于听阈40dB 频率为1000Hz的纯音产生的音调定为1000Mel 音调与频率的近似公式 人耳对不同频段声音的敏感程度不同 通常对低频比对高频更敏感 人耳对语音信号的相位不敏感 17 人耳听觉特性 3 掩蔽效应 Maskingeffect 当两个响度不等的声音作用于人耳时 响度较高的频率成分的存在会影响到对响度较低的频率成分的感受 使其变得不易察觉 这种现象称为掩蔽效应 由于频率低的声音在内耳耳蜗基底膜上传播的距离远于频率较高的声音 故一般来说 低音容易掩蔽高音 而高音掩蔽低音较难 掩蔽会造成因一个声音的存在 而使另一个声音的听阈上升 18 人耳听觉特性 4 临界带宽 Criticalband 噪声的存在会影响到纯音的接收 即对纯音产生掩蔽 为了描述这种掩蔽效果 引入了临界带宽的概念 一个纯音可以被以它为中心频率 且具有一定频带宽度的连续噪声所掩蔽 这个纯音处于刚刚能被听到的临界状态 在这一频带内的噪声功率等于该纯音的功率 即称这一带宽为临界带宽 可以通过实验测得 临界带宽的单位可以用Bark来表示 在20Hz 16kHz范围内的声音可以划分为24个Bark 粗略地讲 一个临界带宽大约相当于耳蜗基底膜上1 5mmc长 或对应大约1200根听神经纤维 临界带宽编号Z Bark 与频率f Hz 之间的关系可以近似表示为 19 B 20 21 现有标准 宽带音频ISO MPEGI 1991 SB Sub BandDBA DynamicBitAllocationISO MPEG2 1993 将采样率扩充到16 22 05 24KHZ 带宽分别为7 5 10 3 11 25KHz 22 PASC MPEG1Layer1 PrecisionAdaptiveSub bandCoding PASC MPEG1 Layer1 飞利浦公司用于DCC DigitalCompactCassette 的算法直接量化需要的数据率采样频率 32kHz 44 1kHz 48kHz 16bit Sampling编码速率 48 16 768kb s立体声1536kb sDCC标准最高记录频率为48kHz 可用8条轨迹记录信号 允许的传输速率 384kb s 23 PASCCoding 将全频带信号划为32个子带将输入序列中连续512个样点数据滤波 输出32个子带样点分块 当每子带中的数据达到12个 即作为一个数据处理单元进行处理 总共32 12 384数据 选择比例因子 找出每一子带12个数据中的最大值 作为本子带的比例因子 用6比特量化编码输出 计算各子带内信号的能量 据以确定相应的量化比特数利用人的听觉感知特性 更经济合理地分配好有限的编码比特 以免做 即使编码了 人耳也听不见 的无用功 对子带中的每个样点进行PCM编码 24 PASC编码原理 按以上PASC在384kb s的码率下具有很高的质量 实测钢琴 吉它的重建波形与原始波形之间看不出差别 而频率较低的鼓声也只有轻微的差异 MPEGLayerIII 25 现有标准 宽带语音 MLT ModulatedLappedTransformThealgorithmisbasedontransformtechnology usingamodulatedlappedtransform MLT Itoperateson20msframes 320samples ofaudio Becausethetransformwindow basisfunctionlength is640samplesanda50 320samples overlapisusedbetweenframes theeffectivelook aheadbuffersizeis20ms Hencethetotalalgorithmicdelayof40msisthesumoftheframesizepluslook ahead Allotherdelaysareduetocomputationalandnetworktransmissiondelays 26 27 现有标准 嵌入式语音编码 现有标准 嵌入式语音编码G 729EV TDBWE TimeDomainBandwidthExtensionTDAC TimeDomainAliasingCancellation G 729EV G 729basedEmbeddedVariablebit ratecoder An8 32kbit sscalablewidebandcoderbitstreaminteroperablewithG 729Encoderat8kbit sandgeneratebitstreamwithG 729format 10msframesInputsamplingrateis16000Hzandthebitstreamisdividedinto20msframes orsuperframes 28 现有标准 语音编码 B 29 ModelofSpeechProduction 1 2 30 31 LinearPredictiveCodingAlgorithm Equation 3 isthewellknownLPdifferenceequation whichstatesthatthevalueofthepresentoutput maybedeterminedby 3 summingtheweightedpresentinput andaweightedsumofthepastoutputsamples 32 InverseFilterProduceResidue 33 AnalysisandSynthesis OriginalResidue 34 35 Original LPCResidual PitchResidual 36 线性预测语音压缩编码 语音带宽200 3400Hz采样8000Hz8bit 样点量化64kb sPCM由于语音信号的短时时不变特性 可以进行分帧处理180样点 帧一组特征参数 帧 37 语音模型 LPC 的参数 1 声道参数PredictiveCoefficients莱文逊 杜宾 Levinson Durbin 递推算法 2 基音周期PitchPeriod最大幅度差法 AMDF 自相关法 Autocorrelation 3 能量EnergyRMS 4 有声无声判决Voicingvoiced unvoiced 简单的二元激励 比特分配BitAllocation 基音周期 有声 无声判决Pitch Voicing7能量RMS5预测系数PredictiveCoef 41同步Sync1 Total Frame 180Sample 22 5ms 54请计算这个编码器的速率 GovernmentStandardLinearPredictiveCodingAlgorithm LPC 10 ByThomasE Tremain BitRate 2 4kb s 采取什么技术可以进一步降低编码速率 38 参数矢量量化比特分配BitAllocation V UV判决3基音周期Pitch9能量RMS8预测系数PredictiveCoef 24同步Sync1 Total Frame 600Sample 75ms 45请计算这个编码器的速率 BitRate600b s声码器 如何改善声码器合成语音质量 编码过程中 哪些措施会降低合成语音质量 39 SELP语音模型 原始语音分析如下参数 直接合成语音 600bit s 参数不量化 预测冗余参数量化成600b s LSFRMSPitchV UV 40 41 AdaptivePredictiveCoding APC TheAPCscheme asshownintheFigure wasoriginallyproposedbyAtalandSchroeder anditemploysboththeshort termpredictor STP andthepitchorlong termpredictor LTP Theresultantexcitationsignalafterinversefilteringisscalarquantizedonasample by samplebasis APCschemeshavebeenproposedfor16kb sandbelow withvariationsonitstreatmentoftheresidualsignal INMARSAT sStandard Bsystememploysa16kb sAPCandisuseformaritimemobilesystems 42 TheLTPcanbeinterpretedas whereDisthe pitchperiod arethe pitchgain coefficientswhichreflecttheamountofcorrelationbetweenthedistantsamples andistheexcitation B 43 44 Analysis by SynthesisCodingofSpeech Thecodingratesbetween4 8 16kb s namelyanalysis and synthesis A and S schemesandanalysis by synthesis A by S schemes AlthoughA and SsuchasRELP APC ATCandSBChavebeenverysuccessfulatratesaround9 6 16kb s below9 6kb stheycannolongerproducegoodqualityspeech Therearetwomainreasonsfortheirshortcomings 45 Analysis by SynthesisCodingofSpeech thecodedspeechisnotanalyzedtoseeifthecodingprocedureisoperatingefficiently i e thereisnocheck controloverthedistortionsofthereconstructedspeech inadaptiveschemes theerrorsaccumulatedfrompreviousframesarenotconsideredinthecurrentframeofanalysis hencetheerrorspropagateintofollowingframeswithoutanyformofresetting 46 Analysis by SynthesisCodingofSpeech TheA by Smethodisnotuniquetospeechcoding butisageneraltechniqueusedinotherareasofestimationandidentification ThebasicideabehindA by Sisasfollows Firstitisassumedthatthesignalcanbeobservedandrepresentedinsomeform e g timeorfrequencydomain Thenatheoreticalformofthesignalproductionmodelisassumed Themodelhasanumberofparameterswhichcanbevariedtoproducedifferentrangesoftheobservablesignal 47 Analysis by SynthesisCodingofSpeech Toderivearepresentationofthemodelthatisofthesameformasthetruesignalmodel atrialanderrorprocedurecanbeapplied Byvaryingtheparametersofthemodelinasystematicway itispossibletofindasetofparametersthatcanproduceasyntheticsignalwhichmatchestherealsignalwithminimumerror assumingthemodelisvalidtobeginwith Therefore whensuchamatchiscalculated theparametersofthemodelareassumedtobetheparametersofthetruesignal 48 Analysis by SynthesisLPC A by SLPC InA by Sschemes particularlyA by SLPCschemes A and S stwomainshortcomingsareincorporated InA by SLPCcodingsystems a closed loop optimizationprocedureisusedtodeterminetheexcitationsignal whichwhenusedtoexcitethe modelfilter producesaperceptuallyoptimumsynthesizedspeechsignal Itisthis closed loop approachwhichenablestheseA by SLPCschemestobefarmoresuccessfulat4 8 9 6kb sthanconventionalA and SschemessuchasAPCandRELP Analysis by SynthesisLPC A by SLPC ThiscoderminimizestheerrorbetweentheoriginalandthesynthesizedsignalaccordingtoasuitableerrorcriterionbyvaryingtheexcitationsignalandtheSTPandLTPfilters Thisisachievedviaasequentialprocedure Firstthetime varyingfilterparametersaredetermined thenwiththesefixedtheexcitationisoptimized Decoder B 49 PerceptuallyWeightedFilter TheA by SLPCcoderminimizestheerrorbetweentheoriginalsignalandsynthesizedsignalaccordingtoasuitablecriterion However atlowbitratesthereisonlyoneorlessbitpersample thusitismoredifficulttomatchcloselythewaveform Whatisrequiredisanerrorcriterionwhichismoreinsympathywiththehumanperceptioncriterion Althoughmuchworkonauditoryperceptionisinprogress nosatisfactoryerrorcriterionhasyetemerged Inthemeantime howeverapopularbutnottotallysatisfactorymethodistheuseofaperceptuallyweightingfilterinA by Sschemes Thisweightingfilterisgivenby Theeffectofthefactordoesnotalterthecenterformantfrequency butjustbroadensthebandwidthoftheformantsbygivenbywhereisthesamplingfrequencyinhertz 50 PerceptuallyWeightedFilter Theamountofde emphasisiscontrolledby whichintroducesabroadeningeffectandmustliebetween0and1 Themostsuitablevalueofisselectedsubjectivelybylisteningtests andfor8kHzsamplingisusuallyaround0 8 0 9 Weightingfilterspectracomparedwiththeoriginalspeechenvelope 51 A by SLPCCodingSchemewithPerceptuallyWeightedFilter 52 SimpleStyle B 53 TheDOD4 8kb sStandard CELP ProposedFederalStandard1016 JosephP Campbell Jretc CELPAlgorithmDescriptionCELPcodingisaframe orientedtechniquethatbreaksasampledinputsignalintoblocksofsamples i e vectors thatareprocessedasoneunit CELPcodingisbasedonA by Ssearchprocedures perceptuallyweightedvectorquantization VQ andlinearprediction LP A10thorderLPfilterisusedtomodelthespeechsignal sshort termformantstructure Long termsignalperiodicityismodeledbyanadaptivecodebookVQ alsocalledpitchVQbecauseitoftenfollowsthespeaker spitchinvoicedspeech Theerrorfromtheshort termLPandpitchVQisvectorquantizedusingafixedstochasticcodebook Theoptimalscaledexcitationvectorsfromtheadaptiveandstochasticcodebooksareselectedbyminimizingatimevarying perceptuallyweighteddistortionmeasurethatimprovessubjectivespeechqualitybyexploitingmaskingpropertiesofhumanhearing 54 The8kHzsamplerateanda30msframesizewithfour7 5mssubframesareused CELPanalysisconsistsofthreebasicfunction CELPsynthesisconsistsofthecorrespondingthreesynthesisfunctionsperformedinreverseorderwiththeoptionaladditionofafourthfunctioncalledapostfilter toenhancetheoutputspeech ThetransmittedCELPparametersare short termlinearpredictionlong termadaptivecodebooksearchstochasticcodebooksearch thestochasticcodeindexandgain theadaptivecodebookindexandgain and10linespectralparameters LSP short termlinearprediction 55 Voiced Pitch Unvoiced WhiteNoise Synthesizer 56 Analyzer Techniques 57 58 CodeBookSearchMethods Thesearchproceduresforthestochasticandadaptivecodebooksarevirtuallyidentical differingonlyintheircodebooksandtargetvectors Toreducecomputation asequentialtwo stagesearchofthecodebooksisperformed Thetargetforthefirststageadaptivecodebooksearchistheweightedlinearpredictionresidualplusencodingerrorsintroducedinpreviousframesthataffectatthepresentframe ThesecondstagestochasticcodebooksearchtargetisthefirststagetargetminusthefilteredadaptivecodebookVQexcitation LettheL 60dimensionalrowvectors andrepresenttheoriginalspeechsignal thesyntheticspeechsignal andtheweightederrorsignal respectively Letvrepresenttheexcitationvectorbeingsearchedforinthepresentstageandletubetheexcitationvectorforthepreviousstage TheexcitationsequenceforanNsizecodebookwithinasub frameofsizeLischaracterizedbyacodebookindexi andacorrespondingoptimizedgainparameter Theexcitationvectorcanbewrittenas 3 1 codebookvector superscript codebookindexofthecodebookvector 59 LetHandWareL Lmatriceswhosej throwscontainthetruncatedimpulseresponsecausedbyaunitimpulseoftheLPfilteranderrorweightingfilter respectively ThesyntheticspeechcanbeexpressedastheLPfilter szeroinputresponse plustheconvolutionoftheLPfilter sexcitationandimpulseresponse 3 2 Theweightederrorsignalis 3 3 60 Target 3 4 Thus theweightederror isthetargetminusthescaledfilteredcodeword 3 5 filteredcodeword 3 6 LetrepresentthenormortotalsquarederrorforcodewordI 3 7 T transpose isafunctionofboththegainfactorandtheindexi theoptimalgaincanbecomputedbysettingthederivativeof withrespecttotheunknowngainvaluetozero 3 8 Therefore theoptimumgainistheratioofthecrosscorrelationofthetargetandfilteredcodewordtotheenergyofthefilteredcodeword 3 9 Thegaincannowbequantizedtojointlyoptimizethesearchforgainandindex 3 10 OurobjectiveistominimizethesquarederroratthereceivergivenbyEq 3 7 with substitutedfor Minimizing withrespectto isequivalenttomaximizingthenegativeofthelasttwotermsinEq 3 7 thefirstterm isindependentofthecodeword Thiscorrespondstomaximizingthematchscore 3 11 Ifthegainquantizationisignored Eq 3 9 canbesubstitutedinEq 3 11 andthematchscoreapproximationisthefamiliarnormalizedsquaredcrosscorrelation 3 12 Thus asshowninEq 11 thecodebooksearchprocedurefortheMSPEexcitationsequenceforA by Sweightedgain shapevectorquantizationistofindthecodeword andgain thatmaximizethematchscore Toreducecomputationalcomplexity thesearchesareperformedsequentially firstusingtheadaptivecodebookanditstarget followedbythestochasticcodebookanditstarget Theoptimalexcitationvectorareentirelycharacterizedbyeachoftheirindicesandcorrespondinggainfactors 61 target Foragivenvalueofi theoptimalgaincanbecomputedbysettingthederivativeofwithrespecttotheunknowngainvaluetozero 3 8 Therefore theoptimumgainistheratioofthecrosscorrelationofthetargetandfilteredcodewordtotheenergyofthefilteredcodeword 3 9 Thegaincannowbequantizedtojointlyoptimizethesearchforgainandindex 3 10 OurobjectiveistominimizethesquarederroratthereceivergivenbyEq 3 7 withsubstitutedfor Minimizingwithrespecttoiisequivalenttomaximizingthenegativeofthelasttwoterms