




版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领
文档简介
LLM4SR:ASurveyonLargeLanguageModelsforScientific
Research
ZIMINGLUO∗
,UniversityofTexasatDallas,USA
arXiv:2501.04306v1[cs.CL]8Jan2025
ZONGLINYANG∗
,NanyangTechnologicalUniversity,Singapore
ZEXINXU,
UniversityofTexasatDallas,USA
WEIYANG,
UniversityofTexasatDallas,USA
XINYADU,
UniversityofTexasatDallas,USA
Inrecentyears,therapidadvancementofLargeLanguageModels(LLMs)hastransformedthelandscapeofscientificresearch,offeringunprecedentedsupportacrossvariousstagesoftheresearchcycle.ThispaperpresentsthefirstsystematicsurveydedicatedtoexploringhowLLMsarerevolutionizingthescientificresearchprocess.WeanalyzetheuniquerolesLLMsplayacrossfourcriticalstagesofresearch:hypothesisdiscovery,experimentplanningandimplementation,scientificwriting,andpeerreviewing.Ourreviewcomprehensivelyshowcasesthetask-specificmethodologiesandevaluationbenchmarks.Byidentifyingcurrentchallengesandproposingfutureresearchdirections,thissurveynotonlyhighlightsthetransformativepotentialofLLMs,butalsoaimstoinspireandguideresearchersandpractitionersinleveragingLLMstoadvancescientificinquiry.Resourcesareavailableatthefollowingrepository:
/du-nlp-lab/LLM4SR.
CCSConcepts:•Computingmethodologies→Naturallanguageprocessing;•Generalandreference→Surveysandoverviews.
AdditionalKeyWordsandPhrases:LargeLanguageModels,ScientificHypothesisDiscovery,ExperimentPlanningandImplementation,AutomatedScientificWriting,PeerReviewGeneration
ACMReferenceFormat:
ZimingLuo,ZonglinYang,ZexinXu,WeiYang,andXinyaDu.2025.LLM4SR:ASurveyonLargeLanguageModelsforScientificResearch.ACMComput.Surv.1,1(January2025),
37
pages.
/10.1145/
nnnnnnn.nnnnnnn
Automating
Research
Process§3.3
Draftingand
Writing
§4.4
CitationText
Generation
§4.2
RelatedWork
Generation
§4.3
Scientific
Hypothesis
Discovery
§2
Peer
Reviewing§5
OptimizingExperimentDesign§3.2
PaperWriting§4
ExperimentPlanning&Implementation
§3
Fig.1.Schematicoverviewofthescientificresearchpipelinecoveredinthissurvey.Thiscyclicalprocessbeginswithscientifichypothesisdiscovery,followedbyexperimentplanningandimplementation,paperwriting,andfinallypeerreviewingofpapers.Theexperimentplanningstageconsistsofoptimizingexperimentdesignandexecutingresearchtasks,whilethepaperwritingstageconsistsofcitationtextgeneration,relatedworkgeneration,anddrafting&writing.
*Bothauthorscontributedequallytothiswork.
Authors’ContactInformation:
ZimingLuo
,ziming.luo@,UniversityofTexasatDallas,Dallas,Texas,USA;
ZonglinYang
,zonglin001@.sg,NanyangTechnologicalUniversity,Singapore,Singapore;
ZexinXu,
zexin.xu@,UniversityofTexasatDallas,Dallas,Texas,USA;
WeiYang,
wei.yang@,UniversityofTexasatDallas,Dallas,Texas,USA;
XinyaDu
,xinya.du@,UniversityofTexasatDallas,Dallas,Texas,USA.
2025.ACM1557-7341/2025/1-ART
/10.1145/nnnnnnn.nnnnnnn
Preprint.
2LuoandYangetal.
1Introduction
“IfIhaveseenfurther,itisbystandingontheshouldersofgiants.”
—IsaacNewton
ThescientificresearchpipelineisatestamenttotheachievementsoftheEnlightenmentinsystematicinquiry
[17,
58,
58
].Inthistraditionalparadigm,scientificresearchinvolvesaseriesofwell-definedsteps:researchersstartbygatheringbackgroundknowledge,proposehypotheses,designandexecuteexperiments,collectandanalyzedata,andfinallyreportfindingsthroughamanuscriptthatundergoespeerreview.Thiscyclicalprocesshasledtogroundbreakingadvance-mentsinmodernscienceandtechnology,yetitremainsconstrainedbythecreativity,expertise,andfinitetimeandresourcesavailableinherenttohumanresearchers.
Fordecades,thescientificcommunityhassoughttoenhancethisprocessbyautomatingaspectsofscientificresearch,aimingtoincreasetheproductivityofscientists.Earlycomputer-assistedresearchcandatebacktothe1970s,introducingsystemssuchasAutomatedMathematician
[74,
75
]andBACON
[71
],whichshowedthepotentialofmachinestoassistinspecializedresearchtasksliketheoremgenerationandempiricallawidentification.Morerecently,systemssuchasAlphaFold
[62
]andOpenFold
[4
]haveexemplifiedpioneeringeffortstoautomatespecificresearchtasks,significantlyspeedingupscientificprogressintheirrespectivedomainsbythousandsoftimes.YetitwasonlywiththeadventoffoundationmodelsandtherecentexplosioninLargeLanguageModels(LLMs)
[2,
154
]thatthevisionofcomprehensiveAIassistanceacrossmultipleresearchdomainsbecamerealistic
[190
].
TherecentyearshavewitnessedremarkableadvancementsinLLMs,transformingvariousfieldsofAIandNaturalLanguageProcessing(NLP).Thesemodels,suchasGPT-4
[2
]andLLaMA
[154
],havesetnewbenchmarksinunderstanding,generatingandinteractingwithhumanlanguage.Theircapabilities,enhancedbymassivedatasetsandinnovativearchitectures,nowextendbeyondconventionalNLPtaskstomorecomplexanddomain-specificchallenges.Inparticular,theabilityofLLMstoprocessmassiveamountsofdata,generatehuman-liketext,andassistincomplexdecision-makinghascapturedsignificantattentioninthescientificcommunity
[92,
141
].ThesebreakthroughssuggestthatLLMshavethepotentialtorevolutionizethewayscientificresearchisconducted,documented,andevaluated
[156,
165,
174
].
Inthissurvey,weexplorehowLLMsarecurrentlybeingappliedacrossvariousstagesofthescientificresearchprocess.Specifically,weidentifyfourgeneraltaskswhereLLMshavedemonstratednotablepotential.Webeginbyexploringtheirapplicationinscientifichypothesisdiscovery,whereLLMsleverageexistingknowledgeandexperimentalobservationstosuggestnovelresearchideas.Thisisfollowedbyareviewoftheircontributionstoexperimentplanningandimplementation,whereLLMsaidinoptimizingexperimentaldesign,automatingworkflows,andanalyzingdata.Wealsocovertheiruseinscientificwriting,includingthegenerationofcitations,relatedworksections,andevendraftingentirepapers.Finally,wediscusstheirpotentialinpeerreview,whereLLMssupporttheevaluationofscientificpapersbyofferingautomatedreviewsandidentifyingerrorsorinconsistencies.Foreachofthesetasks,weprovideacomprehensivereviewofthemethodologies,benchmarks,andevaluationmethods.Moreover,thesurveyidentifiesthelimitationsofeachtaskandhighlightsareasneedingimprovement.ByanalyzingthevariousstagesoftheresearchcyclewhereLLMscontribute,thissurveycaninspireresearcherstoexploreemergingconcepts,developevaluationmetrics,anddesigninnovativeapproachestointegrateLLMsintotheirworkflowseffectively.
ComparisonwithExistingSurveys.ThissurveyprovidesabroaderandmorecomprehensiveperspectiveontheapplicationsofLLMsacrosstheentirescientificresearchcyclecomparedtopriorspecializedstudies.Forexample,Zhangetal.
[187
]reviewover260LLMsinscientificdiscovery
Preprint.
Preprint.
LLM4SR:ASurveyonLargeLanguageModelsforScientificResearch3
Literature-based
Discovery(§2.2.1)
InductiveReasoning(§2.2.2)
MainTrajectory(§2.3.1)
OtherMethods(§2.3.2)
History(§
2.2)
LBD[47,151],DBLP[155],LinkPredictionModels[152,160,171]
Norton[113],Yangetal.[175],Yangetal.[173],Zhongetal.[191],Zhuetal.[194],Wangetal.[163],Qiuetal.[120]
SciMON[159],MOOSE[174],MCR[145],Qi[119],
FunSearch[130],ChemReasoner[146],HypoGeniC[193],
Scientific
HypothesisDiscovery(§
2)
ResearchAgent[9],LLM-SR[140],SGA[105],AIScientist[103],MLR-Copilot[84],IGA[141],SciAgents[41],Scideator[121],
DevelopmentofMethods(§2.3)
MOOSE-Chem[176],VirSci[148],CoI[77],Nova[49],CycleResearcher[167],SciPIP[164]
Socraticreasoning[30],IdeaSynth[118],HypoRefine[96],LDC[80]
DiscoveryBench[108],DiscoveryWorld[57]
SciMON[159],Tomato[174],Qietal.[119],Kumaretal.[68],Tomato-Chem[176] Benchmarks(§2.4)
Evaluation(§2.5)
LLM-based/Expert-basedEvaluation;DirectEvaluation/Reference-basedEvaluation;
DirectEvaluation/Comparison-basedEvaluation;RealExperimentEvaluation
HuggingGPT[136],CRISPR-GPT[52],ChemCrow[15],Coscientist[14],LLM-RDF[131],AutoGen[168],Lietal.[81],Lietal.[90]
OptimizingExperi-
LargeLanguageModels(LLMs)forScientificResearch
mentalDesign(§
3.2)
DataPreparation
(§3.3.1)
ExperimentPlanning
andImple-mentation(§
3)
Clearning[21,185],Labeling[153],FeatureEngineering[46],Synthesis[82,85,98]
ExperimentExecution
——andWorkflow
Automation(§3.3.2)
DataAnalysisand
ChemCrow[15],Coscientist[14],Wangetal.[157],Ramosetal.[124],ChatDrug[99],DrugAssist[179],ESM-1b[128],ESM-2[95],
FerruzandHöcker[35],Heetal.[44]
AutomatingExperi-mentalProcess(§3.3)
Singhetal.[143],Lietal.[79],MentalLLaMA[172],
Interpretation(§3.3.3)
Daietal.[27],Rasheedetal.[126],Zhaoetal.[188],Oliveretal.[114]
TaskBench[137],DiscoveryWorld[57],MLAgentBench[54],AgentBench[100],Spider2-V[16], DSBench[61],DS-1000[70],CORE-Bench[142],SUPER[13],MLE-Bench[20],LAB-Bench[72],
ScienceAgentBench[24]
Xingetal.[170],AutoCite[161],BACO[40],GuandHahnloser[43],Jungetal.[63]
Benchmarks&
Evaluation(§3.4)
CitationText
Generation(§4.2)
Zimmermannetal.[197],Agarwaletal.[3],Huetal.[50],Shietal.[138],Yuetal.[181],Susnjaketal.[150],LitLLM[3],HiReview[50],Nishimuraetal.[112]
RelatedWork
PaperWriting(§
4)
Generation(§4.3)
Augustetal.[8],SCICAP[48],PaperRobot[160],Ifarganetal.[56],CoAuthor[73],AutoSurvey[165],AIScientist[103]
ALCE[38],CiteBench[37],SciGen[111],SciXGen[22]
DraftingandWriting(§4.4)
Benchmarks&
Evaluation(§4.5)
ReviewRobot
[162]
,Reviewer2
[39]
,SWIF2T
[18]
,SEA
[180]
,MARG
[28]
,MetaGen
[11]
,Kumaretal.
[67]
,MReD
[135]
,CGI2
[184]
,CycleReviewer
[167]
AutomatedPeer
ReviewingGeneration(§
5.2)
PaperMage[101],CocoSciSum[29]
ReviewerGPT[97],PaperQA2[144],Scideator[122]
ReviewFlow[149],CARE[198],DocPilot[110]
Information
Summarization
PeerReview-ing(§
5)
ErrorDetection&
QualityVerification
LLM-assistedPeerReviewWorkflows(§
5.3)
Benchmarks&
Evaluation(§
5.4)
ReviewWritingSupport
MOPRD
[94]
,ORSUM
[184]
,MReD
[135]
,PeerSum
[78]
,NLPeer
[33]
,PeerRead
[65]
,ASAP-Review
[183]
,ReviewCritiqe
[32]
,Reviewer2
[39]
Fig.2.Themaincontentflowandcategorizationofthissurvey.
acrossvariousdisciplines,focusingprimarilyontechnicalaspectssuchasmodelarchitecturesanddatasets,withoutsituatingtheirroleswithinthebroadercontextoftheresearchprocess.Similarly,othersurveystendtoadoptnarrowerscopes,examiningspecificcapabilitiesofLLMsforgeneralapplications,suchasplanning
[55
]orautomation
[158
],ratherthantheirfocusedutilityinscientificresearchworkflows.Additionally,someworksaddressgeneralapproachesrelevanttospecificresearchstagesbutarenotexclusivelycenteredonLLMs,suchasrelatedworkandcitationtext
4LuoandYangetal.
Preprint.
generation
[89
]orpeerreviewprocesses
[33
].Incontrast,thissurveyintegratesthesefragmentedperspectives,providingaholisticanalysisofLLMs’contributionsacrossthescientificworkflowandhighlightingtheirpotentialtoaddressthediverseandevolvingdemandsofmodernresearch.
OrganizationofthisSurvey.AsillustratedinFigure
2
,thestructureofthissurveyisasfollows:§
2
coversLLMsforscientifichypothesisdiscovery,includinganoverviewofmethodologiesandkeychallenges.§
3
focusesonexperimentplanningandimplementation,highlightinghowLLMscanoptimizeandautomatetheseprocesses.§
4
delvesintoautomatedpaperwriting,includingcitationandrelatedworkgeneration,while§
5
exploresLLM-assistedpeerreview.Foreachtopic,thesurveyconcludeswithasummaryofcurrentchallengesandfuturedirectionsinthisrapidlyevolvingfield.
2LLMsforScientificHypothesisDiscovery
2.1Overview
Beforetheemergenceofthefield“LLMsforscientifichypothesisdiscovery”,themostrelatedpreviousresearchdomainsare“literature-baseddiscovery”and“inductivereasoning”.Wefirstsummarizetheresearchinthetworelateddomains(ashistory),thensummarizethemethods,benchmarks,evaluationdevelopmenttrends,andimportantprogress,andfinallyconcludewiththemainchallengesinthediscoverytask.
2.2HistoryofScientificDiscovery
UsingLLMstogeneratenovelscientifichypothesesisanewresearchtopic,mostlyoriginatingfromtworelatedresearchdomains,whichare“literature-baseddiscovery”and“inductivereasoning”.
2.2.1Literature-basedDiscovery.Literature-baseddiscovery(LBD)wasfirstproposedbySwanson
[151
].Thecentralideaisthat“knowledgecanbepublic,yetundiscovered,ifindependentlycreatedfragmentsarelogicallyrelatedbutneverretrieved,broughttogether,andinterpreted.”Therefore,howtoretrievepublicknowledgethatcanbebroughttogethertocreatenewknowledgeremainsachallenge.
Swanson
[151
]proposeaclassicformalizationofLBD,whichisthe“ABC”modelwheretwoconceptsAandCarehypothesizedaslinkediftheybothco-occurwithsomeintermediateconceptBinpapers.Morerecentworkhasusedwordvectors
[155
]orlinkpredictionmodels
[152,
160,
171]
todiscoverlinksbetweenconceptstocomposehypotheses.
However,classicLBDmethodsdonotmodelcontextsthathumanscientistsconsiderintheideationprocess,andarelimitedtopredictingpairwiserelationsbetweendiscreteconcepts
[47
].Toovercometheselimitations,Wangetal.
[159
]makethefirstattempttogroundLBDinanaturallanguagecontexttoconstrainthegenerationspace,andalsousegeneratedsentencesasoutputinsteadofonlypredictingrelationsasinthetraditionalLBD.
AnotherlimitationofLBDisthatithaslongbeenthoughtofasonlybeapplicabletoaveryspecific,narrowtypeofhypothesis
[159]
.However,recentprogressinscientificdiscoveryindicatesthatLBDmighthaveamuchwiderapplicablescope.Particularly,Yangetal.
[174
]andYangetal.
[176
]discussextensivelywithsocialscienceandchemistryresearcherscorrespondingly,andfindthatmostexistingsocialscienceandchemistrypublishedhypotheses(insteadofonlyanarrowtypeofhypotheses)canbeformulatedinaLBDpattern.Itprobablyindicatesthatfuturehypothesesinsocialscienceandchemistrytobepublishedcanalsoresultfrom(correct)linkagesandassociationsofexistingknowledge.
2.2.2InductiveReasoning.Inductivereasoningisaboutfindingageneral“rule”or“hypothesis”thathasawideapplicationscopefromspecific“observations”
[175
].Forexample,Geocentrism,
LLM4SR:ASurveyonLargeLanguageModelsforScientificResearch5
Preprint.
Heliocentricism,andNewton’sLawofGravityareallproposed“rules”basedonthe“observations”ofthemovementsofstarsandplanets.Scientificdiscoveryisadifficulttaskofinductivereasoningtoanextreme,whereeach“rule”isanovelscientificfinding.
Thephilosophyofsciencecommunityhassummarizedthreefundamentalrequirementsfora“rule”frominductivereasoning
[113
],whichare(1)“rule”shouldnotbeinconflictwith“observa-tions”;(2)“rule”shouldreflectthereality;(3)“rule”shouldpresentageneralpatternthatcanbeappliedtoalargerscopethanthe“specific”observations,coveringnewinformationnotexistingintheobservations.Previouslyinductivereasoningresearchismainlyconductedbythe“inductive logicprogramming”community
[26
],whichusesformallanguageandsymbolicreasoners.Yangetal.
[173
]firstworkongenerativeinductivereasoningintheNLPdomain,whichistogeneratenaturallanguagerulesfromspecificnaturallanguageobservationswithlanguagemodels,introduc- ingtherequirementsoninductivereasoningfromthephilosophyofsciencecommunity.Motivatedbytheempiricalexperiencethatlanguagemodelstendtogeneratevagueandnotspecificrules,theyadditionallyproposethefourthrequirement:(4)“rule”shouldbeclearandinenoughdetail.Thefourthrequirementmighthavebeenoverlookedbythephilosophyofsciencecommunitysince it’stooobvious.Motivatedbytherequirements,Yangetal.
[173
]designanoverly-generation-then-filteringmechanism,leveraginglanguagemodelstofirstgeneratemanypreliminaryrulesandthenfilterthosedonotsatisfytherequirements.Thenmethodsaredevelopedtouseself-refinetoreplacefilteringandusemorereasoningstepsforbetterrules
[120,
163,
191,
194
].However,the“rules”thislineofworkstrytoinduceareeitherknownknowledge,ornotscientificknowledgebutsynthesizedpatterns.
Yangetal.
[174
]makethefirstattempttoextendtheclassicinductivereasoningtasksetting(todiscoverknown/syntheticknowledge)intoarealscientificdiscoverysetting:toleverageLLMstoautonomouslydiscovernovelandvalidsocialsciencescientifichypothesesfromthepubliclyavailablewebdata.Specifically,theycollectnews,businessreviews,andWikipediapagesonsocialscienceconceptsasthewebdatatodiscoverhypothesis.
Majumderetal.
[107,
108
]furtherproposetheconceptof“data-drivendiscovery”,whichistodiscoverhypothesesacrossdisciplineswithallthepublicexperimentaldataontheweb(andprivateexperimentaldataathand).Theirmotivationisthatthepotentialofthelargeamountofpubliclyavailableexperimentaldatahasnotbeenfullyexploitedthatlotsofnovelscientifichypothesescouldbediscoveredfromtheexistingdata.
2.3DevelopmentofMethods
Amongthemethodsdevelopedforscientificdiscovery,thereisoneclearmethoddevelopmenttrajectory.Webeginbyintroducingthistrajectory,followedbyanexplorationofothermethods.
2.3.1MainTrajectory.Ingeneral,thismethoddevelopmenttrajectoryforscientificdiscoverycanbeseenasincorporatingmorekeycomponentsintothemethods.Table
1
summarizesthekeycomponentsweidentifyasimportantandindicateswhethereachmethodincorporatesthem.Specifically,theyare“strategyofinspirationretrieval”,“noveltychecker”,“validitychecker”,“claritychecker”,“evolutionaryalgorithm”,“leverageofmultipleinspiration”,“rankingofhypothesis”,and“automaticresearchquestionconstruction”.Here,each“keycomponent”referstoadetailedanduniquemethodologythathasproveneffectiveforscientificdiscoverytasks.Weexcludebroadgeneralconceptsthatmayintuitivelyseemhelpfulbutit’snotclearhowaspecificmethodfromtheconceptcanbeeffectiveforthistask(e.g.,toolusage).Next,weintroducethesekeycomponents.Foreachkeycomponent,weuseoneortwoparagraphstogiveashortoverview,summarizingitsdevelopmenttrace.ThereferenceinformationforeachmethodmentionedinthissectioncanbefoundinTable
1.
6LuoandYangetal.
Preprint.
InspirationRetrievalStrategy.Inadditiontorelyingonbackgroundknowledge,literature-baseddiscovery(LBD)facilitatestheretrievalofadditionalknowledgeasasourceofinspirationforformulatingnewhypotheses.SciMON
[159
]firstintroducestheconceptsofLBDtothediscoverytask,demonstratingthatnewknowledgecanbecomposedoflinkageofexistingknowledge.Itisvitalthattheinspirationshouldnotbeknowntoberelatedtothebackgroundbefore,oratleastshouldnotbeusedtoassociatewiththebackgroundinaknownway
[176
].Otherwise,thehypothesiswouldnotbenovel.
Inspiredbythe“ABC”modelinclassicLBDformalization,givenabackgroundknowledge,SciMONretrievessemanticallysimilarknowledge,knowledgegraphneighbors,andcitationgraphneighborsasinspirations.Specifically,twoknowledgeareidentifiedas“semanticallysimilar”iftheirembeddingsfromSentenceBERT
[127
]havehighcosinesimilarity;Theknowledgegraphtheybuiltfollowsa“[method,used-for,task]”format.ResearchAgentstrictlyfollowsthe“ABC”modelbyconstructingaconceptgraph,wherealinkrepresentsthetwoconnectedconceptnodeshaveappearedinthesamepaperbefore.Itretrievesinspirationconceptsthatareconnectedwiththebackgroundconceptsontheconceptgraph(conceptco-occurence).Scideatorretrievesinspirationpapersbasedonsemanticmatching(semanticscholarAPIrecommendations)andconceptmatch-ing(paperscontainingsimilarconceptsinthesametopic,samesubarea,anddifferentsubarea).SciPIP
[164
]retrievesinspirationsfromsemanticallysimilarknowledge(basedonSentenceBERT),conceptco-occurence,andcitationgraphneigbors.Itproposesfilteringmethodstofilternotusefulconceptsforconceptco-occurenceretrieval.
Differentfromselectingsemanticorcitationneighborsasinspirations,SciAgentsrandomlysampleanotherconceptthatisconnectedwiththebackgroundconceptinacitationgraph(viaalongorshortpath)astheinspiration.
MOOSE
[174
]proposestouseLLM-selectedinspirations:giventheresearchbackgroundandsomeinspirationcandidatesinthecontext,andaskanLLMtoselectinspirationsfortheresearchbackgroundfromthecandidates.ThenMOOSE-Chem
[176
]alsoadoptsit.MOOSE-Chemassumesthataftertrainingonhundredsofmillionsofscientificpapers,themostadvancedLLMsmightalreadyhaveacertainlevelofabilitytoidentifytheinspirationknowledgeforthebackgroundtocomposeanoveldiscoveryofknowledge.MOOSE-Chemanalyzesthisassumptionbyannotating51chemistrypaperspublishedin2024(whichareonlyavailableonlinein2024)withtheirbackground,inspirations,andhypothesis,andseewhetherLLMswithtrainingdataupto2023canretrievetheannotatedinspirationsgivenonlythebackground.Theirresultsshowaveryhighretrievalrate,indicatingthattheassumptioncouldbelargelycorrect.ThenNovaalsoadoptsLLM-selectedinspirations,withthemotivationthatleveragingtheLLM’sinternalknowledgetodetermineusefulknowledgefornewideasshouldbeabletosurpasstraditionalentityorkeyword-basedretrievalmethods.
FeedbackModules.Thenextkeycomponentistheiterativefeedbackonthegeneratedhypothesesintheaspectsofnovelty,validity,andclarity.ThesethreefeedbacksarefirstproposedbyMOOSE,motivatedbytherequirementsforahypothesisininductivereasoning
[113,
173
].Thesethreeaspectsareobjectiveenoughtogivefeedback,andeachofthemisessentialforagoodhypothesis.
•NoveltyChecker.Thegeneratedhypothesesshouldbeanovelfindingcomparedtotheexistingliterature.Whenahypothesistendstobesimilartoanexistinghypothesis,feedbackonenhancingitsnoveltycouldbebeneficialforhypothesisformulation.ExistingmethodsfornoveltyfeedbackareallbasedonLLMs.Ingeneral,therearethreewaystoprovidenoveltyfeedback.Thefirstmethodevaluateseachgeneratedhypothesisagainstarelatedsurvey(MOOSE);theseconditerativelyretrievesrelevantpapersforcomparison(SciMON,
LLM4SR:ASurveyonLargeLanguageModelsforScientificResearch7
Preprint.
Table1.DiscoveryMethods.Here“NF”=NoveltyFeedback,“VF”=ValidityFeedback,and“CF”=ClarityFeedback,“EA”=EvolutionaryAlgorithm,“LMI”=LeveragingMultipleInspirations,“R”=Ranking,“AQC”=AutomaticResearchQuestionConstruction.Theorderofmethodsreflecttheirfirstappearancetime.
Methods
InspirationRetrievalStrategyNFVFCFEALMIRAQC
SciMON
[159]
MOOSE
[174]
MCR
[145]
Qi
[119]
FunSearch
[130]
ChemReasoner
[146]
HypoGeniC
[193]
ResearchAgent
[9]
LLM-SR
[140]
SGA
[105]
AIScientist
[103]
MLR-Copilot
[84]
IGA
[141]
SciAgents
[41]
Scideator
[121]
MOOSE-Chem
[176]
VirSci
[148]
CoI
[77]
Nova
[49]
CycleResearcher
[167]
SciPIP
[164]
-
-
Semantic&Concept&CitationNeighbors√
√
√
LLMSelection√
-
√
√
√
√
√
√
√
√
√
-
-√
-
-
-
√
ConceptCo-occurrenceNeighbors√
-
-
-
-√
-
-
--
-
-
--
√
-
RandomSelection√
-
-
√
√
√
√
Semantic&ConceptMatching√
-
-
LLMselection√-√-√
-
-
LLMselection-
-
-
--
-
-
Semantic&Concept&CitationNeighbors-
-
-
-
-
-
-
-
-
√
-
-
-
√
-
√
-
-
√
-
-
√
-
-
-
√
√
√
-
√
-
-
-
√
-
-
-
-
-
√
-
-
-
-
-
-
√
√
√
-
-
-
-
-
-
-
√
-
-
-
√
-
-
-
-
√
-
-
-
-
-
-
-
-
√
√
-
-
-
-
√
√
-
-
-
SciAgents,Scideator,CoI);thethirddirectlyleveragestheinternalknowledgeofLLMsforevaluation(Qi,ResearchAgent,AIScientist,MOOSE-Chem,VirSci).
•ValidityChecker.Thegeneratedhypothesesshouldbevalidscience/engineeringfindingsthatpreciselyreflecttheobjectiveuniverse
[113]
.Arealvalidityfeedbackshouldbefromtheresultsofexperiments.However,itistime-consumingandcostlytoconductexperimentsforeachgeneratedhypothesis.Therefore,currently,validityfeedbackalmostentirelyreliesontheheuristicsofLLMsorothertrainedneuralmodels.TheexceptionsareFunSearch,HypoGeniC,LLM-SR,andSGA.Specifically,FunSearchisaboutgenerat
温馨提示
- 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
- 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
- 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
- 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
- 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
- 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
- 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
最新文档
- 广州新华学院《高等代数BII》2023-2024学年第一学期期末试卷
- 内蒙古交通职业技术学院《数据挖掘案例分析》2023-2024学年第二学期期末试卷
- 甘孜藏族自治州稻城县2024-2025学年数学四年级第二学期期末联考试题含解析
- 云南中医药大学《幼儿园手工与环创》2023-2024学年第一学期期末试卷
- 安徽黄梅戏艺术职业学院《数据结构和算法应用》2023-2024学年第一学期期末试卷
- 昆明学院《视觉与空间设计基础》2023-2024学年第二学期期末试卷
- 揭秘2025保安证考试试题及答案
- 湖南省示范名校2025届高三五校5月适应性考试英语试题试卷含解析
- 甘肃警察职业学院《统计专业综合实训1》2023-2024学年第二学期期末试卷
- 2025年保安证考试成绩提升之道试题及答案
- 2025年高考语文专题复习:标点符号的规范用法 课件
- 附件1:肿瘤防治中心评审实施细则2024年修订版
- 2024-2030年中国自动自攻铆钉行业市场发展趋势与前景展望战略分析报告
- DL∕T 868-2014 焊接工艺评定规程
- 2024年北京中考语文试题及答案
- 幼儿园足球课程实施方案(18篇)
- 【地理】河南省洛阳市强基联盟2023-2024学年高一下学期3月联考试题(解析版)
- 2024届上海市嘉定区高三语文一模试卷(含答案与解析)
- 呼吸衰竭课件新课件
- 北京市海淀区2024年七年级下学期数学期中考试试卷(附答案)
- 污水管网巡查及养护投标方案(技术标)
评论
0/150
提交评论