2024大语言模型变异分析

ReRe欧空局Ariane 波音737Max坠 海湾战争中爱国者飞 当我们说软件是正确的：程序的行为符合

intfoo(inta,intb)returna+

输入 →

Formal(a>=0&&b<=0)||(a<=0&&b>=0)(a>=0&&b>=0&&a+b>=0)(a<=0&&b<=0&&a+b<=

foo(0,1)=1;foo(INT_MAX,1)=ERROR;foo(INT_MAX-1,1)=INT_MAX;foo(INT_MAX,INT_MIN)=-1;我们朴素的愿望：希望测试能变异算子(mutationExecuteExecutetestExecutetest变异算子(mutation变异算子(mutation 1971DeMilloHamlet面向中间表示：Javabytecode,LLVM-Android、MPI基于变异的缺陷自动定位（mutation-basedfaultlocalization）传统定位方法：基于测试覆盖信息对语句排序（spectrum-basedfault基于变异的方法：通过观察变异对测试结果CProteum(108mutationWinMut(IR-JavaMajor[ByReneJavaLanch(IR-based)[ByPITest(CommercialSoftware-artifactInfrastructureRepository(somehowDefects4JManyBugs挑战1：BugDeepMutationICSME-19]、LEAM[ASE-挑战2：挑战2：阻碍变异分析走向工业实践的是可扩展性（Scalability）M个变异，N变异分析的计算复杂度：O(M)+𝑇𝑡𝑜𝑡𝑎𝑙=∑𝑚∈𝑀𝑡𝑠𝑒𝑒𝑑,𝑚+ 𝑡𝑐𝑜𝑚𝑝𝑖𝑙𝑒,𝑚+ ∑𝑛∈𝑁在实际规模的程序中，M挑战3：ii!=

for(inti=0;i<10;

for(inti=

面向挑战1：buggy->面向挑战2：4包括开源模型和商用闭源模型（实验还在扩展中GPT-3.5-TurboGPT-4-TurboAPI23090Java（JavaBaselineDefects4Jv1.20395ConDefects45JavaLEAM[ASE-muBert[ICST- 实缺陷作为Few-shot时间代价（秒平均生成1k等价变异率（按95%置信度和5%误差幅度采样AST节点类型(例如，aba–babfoo(ab)ASTBLEUASTOchiai（语义相似度指标P2：P1few-shotP3：P2P4：P1基于大模型的变异生成：不同的Few-shot我们将所有工具生成的变异数限制为最少的一个analyzebytesttestqualityanalyzebytesttestqualityassessment→mutationfaultlocalization→passorvoidtest_foo(){assert(foo(a,b,c)==RES);EachmutanthasasyntacticTestfoo(inta,intb,Testfoo(inta,intb,intc){intd=a+b;inte=(++d)+c;returne;foo(inta,intb,intc){intd=(++a)+b;inte=d+c;returne;foo(inta,intb,intc){intd=a+b;inte=d+c;returne;foo(inta,intb,intc){intd=(a+1)+b;inte=d+c;returne;returnreturnreturnCyclesrepresentstates,thenumbersinsideareidArrowsrepresentstatetransitionsbyexecutingthelabelstatementintd=a+b;inte=d+c;returne;intd=(a+1)+b;inte=d+c;returnintdintd=(++a)+b;inte=d+c;returnintd=a+inte=(++d)+return

return

Therearealotofredundantcomputation.Forexample,theexecutionfromthebeginningtothestate0 Theinterpreterforkschild-processeswhenencountersamutatedstatement.returnStartwithaprocesscarryingallreturnreturnReplacemutatedstatementswithSSEd=a+b;inte=d+c;returne;intd=interpret(a,b,ADD);inte=Theinterpreterforkschild-processeswhenencountersamutatedstatement.returnStartwithaprocesscarryingallreturnreturnReplacemutatedstatementswithSSEd=a+b;inte=d+c;returne;intd=interpret(a,b,ADD);inte=interpret(d,c,ADD);returne;intd=(a+1)+b;inte=d+c;intd=(++a)+b;inte=d+c;intd=a+inte=(++d)+returnreturnreturnreturnTheredundancyafterthefirstlineisstillalivebeforethestate0Wangetal.,ThelabelbelowistheexecutedexpressioninsideinterpreterWangetal.,ThelabelbelowistheexecutedexpressioninsideinterpreterTheinterpreterTheinterpreteralsocarriesallmutantsItclustersthemutantsofonestatementbyEMS,andforksnewprocessforeach

returnintintd=interpret(a,b,ADD);inte=interpret(d,c,ADD);returne;M1,Instrumentasreturn return

return

2.AccMutinvolves2.AccMutinvolvesalotof AccMutisunabletomergemorereturnM3isSplitcauseitisinareturnTheinterpreterinvocationsinachildprocessisuselessM1andM2aresplitbythevalueofthevariablea,however,whichhasnoeffectonthefinalresult

1 returnreturn

return return Fewer ContinuouslyinterpretabasicblockreturnFewer ContinuouslyinterpretabasicblockreturnreturnDirectlyexecutetheoriginreturnInthechildprocessofaformerOnlycomparethevariableeMergeallthemutantsinasingleAvoidinvokingtheM1,M2,2.Onlycomparethevariablesthatmayaffectthefinal1.Enlargeanalysisscopefromastatementtoabasic3.Avoidunnecessarily3.AvoidunnecessarilyinterpretingmutatedSmallerLLVM-IR中