未来能源研究所-学习如何通过清洁能源政策评估更好地重建

LearningHowtoBuildBack

BetterthroughCleanEnergy

PolicyEvaluation

JosephE.Aldy

WorkingPaper22-15

August2022

ResourcesfortheFuturei

AbouttheAuthor/Authors

JosephE.AldyisauniversityfellowatResourcesfortheFutureandaProfessorofthePracticeofPublicPolicyatHarvard’sKennedySchool.Hisresearchfocusesonclimatechangepolicy,energypolicy,andmortalityriskvaluation.AldyalsocurrentlyservesasthefacultychairoftheRegulatoryPolicyProgramattheHarvardKennedySchool.In2009–2010,heservedasthespecialassistanttothepresidentforenergyandtheenvironment,reportingthroughboththeWhiteHouseNationalEconomicCouncilandtheOfficeofEnergyandClimateChange.

Acknowledgements

FantasticresearchassistancewasprovidedbyMichaelChen,EmilyFry,CharlesHua,MichelleLi,ConnorMcRobert,EmMurdock,KenNorris,SiddShrikanth,andDanStuart.IhavebenefittedfromexcellentfeedbackfromDanielleArostegui,JulieGohlke,WesLook,KevinRennert,MorganRote,BeiaSpiller,andNatashaVidangos,seminarattendeesatFloridaState,MIT,ResourcesfortheFuture,UniversityofHouston,andtheASSAannualconference.ThisresearchhasbeensupportedbyEnvironmentalDefenseFund,HarvardUniversityCenterfortheEnvironment,HKSMossavar-RahmaniCenterforBusinessandGovernment,andHKSCenterforPublicLeadership.

LearningHowtoBuildBackBetterthroughCleanEnergyPolicyEvaluationii

AboutRFF

ResourcesfortheFuture(RFF)isanindependent,nonprofitresearchinstitutioninWashington,DC.Itsmissionistoimproveenvironmental,energy,andnaturalresourcedecisionsthroughimpartialeconomicresearchandpolicyengagement.RFFiscommittedtobeingthemostwidelytrustedsourceofresearchinsightsandpolicysolutionsleadingtoahealthyenvironmentandathrivingeconomy.

Workingpapersareresearchmaterialscirculatedbytheirauthorsforpurposesofinformationanddiscussion.Theyhavenotnecessarilyundergoneformalpeerreview.TheviewsexpressedherearethoseoftheindividualauthorsandmaydifferfromthoseofotherRFFexperts,itsofficers,oritsdirectors.

SharingOurWork

OurworkisavailableforsharingandadaptationunderanAttribution-NonCommercial-NoDerivatives4.0International(CCBY-NC-ND4.0)license.Youcancopyandredistributeourmaterialinanymediumorformat;youmustgiveappropriatecredit,providealinktothelicense,andindicateifchangesweremade,andyoumaynotapplyadditionalrestrictions.Youmaydosoinanyreasonablemanner,butnotinanywaythatsuggeststhelicensorendorsesyouoryouruse.Youmaynotusethematerialforcommercialpurposes.Ifyouremix,transform,orbuilduponthematerial,youmaynotdistributethemodifiedmaterial.Formoreinformation,visit

/licenses/by-nc-nd/4.0/

.

iii

Abstract

TheInfrastructureInvestmentandJobsAct,theCHIPSandScienceAct,andtheInflationReductionActauthorizedandappropriatedunprecedentedspendingandtaxexpenditurestodecarbonizetheAmericaneconomy.Inthespiritof“buildbackbetter,”thispaperexamineshowintegratingevaluationinthedesignandimplementationofthesenewcleanenergypoliciescanfacilitatethelearningnecessaryforpolicymakerstomakepolicybetterovertime.Itdrawslessonsfromtwocasestudies:(1)oninstitutionalizingevaluationbasedontheexperiencewithregulatoryreview,and(2)onconductingevaluationbasedontheresearchliteratureassessingthe2009RecoveryAct’scleanenergyprograms.Thepaperidentifiesinrecentlegislationtheprogramsandtheircharacteristicsamenabletovariousevaluationmethodologies.Thepaper

closeswithrecommendationsforacleanenergyprogramevaluationframework

thatwouldenableimplementationofclimate-orientedlearningagendasunderthe

Evidence-BasedPolicymakingAct.

LearningHowtoBuildBackBetterthroughCleanEnergyPolicyEvaluationiv

Contents

1.Introduction1

2.InstitutionalizingProgramEvaluation:LessonsfromRegulatoryReview3

2.1.DemonstratingtheCompellingNeedforPolicyAction3

2.2.StandardizingEvaluationMethodsandProcess4

2.3.PromotingaCultureforRetrospectiveAnalysisandIterativePolicymaking5

3.ConductingProgramEvaluations:LessonsfromAcademicResearchofthe

AmericanRecoveryandReinvestmentActof20096

3.1.LearningthroughRandomization:TheWeatherizationAssistanceProgram7

3.2.ComparingWinnersandLosers:SmallBusinessInnovationResearchGrants9

3.3.ExploitingStateVariation:StateEnergy-EfficientApplianceRebateProgram11

3.4.ExploitingFormulaAllocations:EmploymentImpactsofCleanEnergy

Programs12

4.PlanningforCleanEnergyProgramEvaluations14

4.1.DevelopCross-cuttingandAgency-specificGuidanceforPerformance

Evaluations14

4.2.IdentifyPriorityOutcomestoEvaluate14

4.3.IdentifyPoliciesandProgramswithSignificantLearningPotential15

4.4.DevelopEvaluationPlansandDataProtocols15

4.5.EnsureEvaluationPlanTransparency16

4.6.PromoteaPerformanceEvaluationCulture17

5.ConclusionsandPolicyImplications18

6.References19

LearningHowtoBuildBackBetterthroughCleanEnergyPolicyEvaluation1

1.Introduction

In2021,theU.S.Governmentpledgedtoreduceitsgreenhousegasemissions

50-52percentbelowtheir2005levelsby2030andtoachieveeconomy-widenet-zeroemissionsby2050.Tomakeprogressontheseemissiongoals,theBidenAdministrationandCongresshaveadvancedanambitiousprogramto“buildbackbetter”throughtheInfrastructureInvestmentandJobsAct,1theCHIPSandScienceAct,2andtheInflationReductionAct.3Theselawsbuildondecadesofcleanenergypolicyatthefederal,state,andlocallevels,including:taxcredits,accelerateddepreciation,taxexemptions,rebates,grants,loans,loanguarantees,andregulatoryandinformationdisclosurerequirements.Inthespiritofbuildbackbetter,integratingprogramevaluationinthedesignandimplementationofnewcleanenergypoliciescanfacilitatethelearningnecessaryforpolicymakerstomakepolicybetterovertime:increasingthelikelihoodofachievingclimategoalsandreducingthecostsofdoingso.

Threekeycharacteristicsoftheclimatechallengeillustratethesignificantvalueinevaluatingcleanenergypolicyperformance.First,transformingthemodernenergyeconomytocombatclimatechangewillrequireunprecedenteddepthandbreadthofpolicyaction.Pastpolicyexperienceslikelyprovideincompleteinsightsforhowtodesignambitiousdecarbonizationpolicies.Acontinuouslearningprocesswillbeneededaswedeploynewtechnologiesandpolicystrategies.Second,manytechnological,environmental,social,andeconomicuncertaintiescharacterizingcleanenergywillberesolvedbypolicypractice.Somepolicieswillturnoutmoreeffectivethanexpected,whileotherslesseffectivethanexpected.Policyexperimentationreducinguncertaintywillprovidethefoundationformakingpolicybetterovertime.Third,thepolicyresponsetoclimatechangewillcontinuetooccurthroughaseriesofbillsandregulationsovertime:annualappropriations;taxextenderpackages;agriculture,energy,andtransportationbills;reconciliationbills;otherlegislation;regulatorystandards,andmore.Iterativepolicyprocessescreateopportunitiesforusinglessonstoinformandimprovefuturepolicydesign.

Understandingthecausalimpactsofpolicy—e.g.,howdidacleanenergypolicydirectlychangeemissions,energyinvestment,employment,publichealth,etc.—iscriticalforimprovingpolicydesignandimplementationovertime.AstheCommissiononEvidence-BasedPolicymaking(2017)noted,“[p]olicymakersmusthavegoodinformationonwhichtobasetheirdecisionsaboutimprovingtheviabilityandeffectivenessofgovernmentprogramsandpolicies.Today,toolittleevidenceisproducedtomeetthisneed”(p.1).Despitethedearthofadequateevidence,theCommissionemphasizedaconstructivepathforward:“[m]oderntechnologyandstatisticalmethods,combinedwithtransparencyandastronglegalframework,createtheopportunitytousedataforevidencebuildinginwaysthatwerenotpossibleinthepast”(p.1).TheFoundationsforEvidence-BasedPolicymakingActof2018

1P.L.117-58.

2P.L.117-167.

3P.L.117-169.

LearningHowtoBuildBackBetterthroughCleanEnergyPolicyEvaluation3

2.InstitutionalizingProgramEvaluation:LessonsfromRegulatoryReview

Since1981,RepublicanandDemocraticAdministrationshaverequiredregulatoryagenciestoestimatetheprospectivebenefitsandcostsoftheirmajorregulatoryproposalsasapartoftheregulatoryreviewprocess.5Environmentalandenergyregulationsrepresentadisproportionateshareoffederalregulatoryproposals.Over2007-2016,theEnvironmentalProtectionAgency(EPA),DepartmentofEnergy,andDepartmentofTransportation(inrulesjointly-issuedwithEPA)issuedmorethanhalfofallmajorfederalregulations(OMB2018).Theseenvironmentalandenergyrulesrepresentmorethan85percentoftheprospectivebenefitsand75percentoftheprospectivecostsofmajorFederalregulations(Aldy2020b).Theexperiencewithregulatoryreviewholdsthreemajorlessonsforinstitutionalizingcleanenergyprogramevaluation.

2.1.DemonstratingtheCompellingNeedforPolicy

Action

Policymakerscancommunicatemoreeffectivelywhyapolicyactionisinournation’sinterestbymarshallingevidenceoftheimpactsofthatpolicyaction.Forexample,thecurrentregulatoryreviewprocessrequiresfederalagenciestodemonstratethattheirregulationsaddressa“compellingneed,suchasmaterialfailuresofprivatemarketstoprotectorimprovethehealthandsafetyofthepublic,theenvironment,orthewell-beingoftheAmericanpeople”(E.O.12866,§1(a)).Inarulemaking,aregulatoryagencyidentifiesthemarketfailure,highlightshowtheproposedregulatoryactionaddressesthemarketfailureandwhyitispreferredtoalternativeapproaches,andshowshowthebenefitsjustifythecosts.The“compellingneed”standardthatmotivatesregulatoryactionswouldreasonablyapplytoanypublicpolicy,includingspendingandtaxexpenditures,thatpromotescleanenergyinvestmenttocombatclimatechange.Spendingandtaxpolicythatdeliveronthesameobjectiveasaregulatoryactionmeritacomparableapproachtoevaluation.

Virtuallyallcleanenergyspendingeffectivelysubsidizesinvestmentinequipmentandcapitalthatcouldbemandatedunderregulatorystandardstoaddressclimatechange-relatedmarketfailures.Forexample,furnaceshavebeensubjecttominimumenergyefficiencystandards,6qualifiedforenergy-efficientappliancerebates(HoudeandAldy

5See:E.O.12291,46FederalRegister13193,February17,1981;andE.O.12866,58FederalRegister51735,October4,1993.

6Referto“EnergyConservationProgramforConsumerProducts:EnergyConservationStandardsforResidentialFurnacesandBoilers,”72FederalRegister65136,November19,2007.

4

2017),andbeeneligiblefortaxcredits.7Windpowerhasbeeneligibleforproductiontaxcredits,§1603grants,and§1705loanguarantees(Aldy2013),andplayedakeyroleindeterminingemissionstandardsunderEPA’sCleanPowerPlan(Fowlieetal.2014).

Justasanalysiscaninformtheselectionanddesignofpreferredregulatoryoptions,evaluationsofspendingandtaxprogramscanenhancepolicymakerunderstandingofthemosteffectiveinstrumentsfordeliveringoncleanenergyobjectives.Producingsuchanalysestaketimeandresources;thus,theregulatoryreviewrequirementsapplyonlytothelargestregulatoryactions—thosewithatleast$100millioninannualeconomicimpacts—wherethevalueofinformationgeneratedislikelytobegreatest.The$100millionimpactthresholdthattriggersafull-blownanalysisofregulatoryimpactsismodestrelativetothesizeofmajorcleanenergytaxandspendingprogramsinrecentlaws(e.g.,theInfrastructureInvestmentandJobsActandtheInflationReductionAct).Theseregulatoryanalysesmatterintheregulatory

developmentprocess:theyinformchangestotheruleaftertheproposalstage,and

theyarerequiredtobesubmittedtoCongresswithallmajorfinalrulesunderthe

CongressionalReviewAct.

2.2.StandardizingEvaluationMethodsand

Process

Theevaluationofcleanenergyprogramscandrawfromexistingguidanceintheregulatoryspace.Theycouldalsodrawfromprogramevaluationproceduresappliedtonon-climatepoliciesinotherpartsofthefederalgovernment,suchastheDepartmentsofHealthandHumanServicesandLabor.Thedevelopmentofstandardproceduresforevaluatingcleanenergyspendingprogramscouldreducethetimeandresourcerequirementsforplanningandexecutingprogramevaluations.Suchstandardizedproceduresandguidancecouldfallunderadepartment’slearningagendaandplandevelopmentundertheEvidence-BasedPolicymakingAct.

Forexample,OMB(2003)issuesguidancetoregulatoryagenciesontheconductofregulatoryimpactanalyses.Theguidanceaddressestheeconomicprinciplesandsomecommoneconomicassumptionsthatshouldinformagencyestimationofbenefitsandcosts.Theguidanceemphasizesboththeexpectedrigorofanalysis—andtheimportanceofrelyingonpeer-reviewedliterature—aswellasthecommunicationoftheresultsoftheanalysistoenableaclearunderstandingbypolicymakers,stakeholders,andthepublic.Suchregulatoryimpactanalysesoftengobeyondsimplytallyingandcomparingbenefitsandcosts;theyalsopresentestimatedemploymentandcompetitivenessimpacts,ancillarybenefitsbeyondthetargetoftherule,aswellasthedistributionanduncertaintycharacterizingtheimpactsoftheregulatoryaction(Aldyetal.2021,Robinsonetal.2016).

Severalregulatoryagencieshavedevelopedtheirownguidancefortheconduct

ofprospectiveregulatoryimpactanalyses,suchasEPA(2014)andDepartmentof

7P.L.111-5,section1121.

LearningHowtoBuildBackBetterthroughCleanEnergyPolicyEvaluation5

HealthandHumanServices(2016).TheDepartmentofTransportation(2021)issuesregularupdatesofitsapproachforvaluingreductionsinmortalityriskthroughitsregulatoryauthorities.TheBidenAdministrationrelaunchedtheinteragencyworkinggrouponthesocialcostofgreenhousegases,whichprovidesestimatesofthesocialcostofcarbon,methane,andnitrousoxidethatcanmonetizethebenefitsofreducinggreenhousegasemissionsthroughregulationandotherFederalactions.8Toimproveunderstandingoftheenvironmentaljusticeimplicationsoffederalinvestments,OMB(2021b)issuedguidanceforhowtocalculateandreportthebenefitsofsuchactionsundertheJustice40Initiative.Theseguidancedocumentstypicallyhaveundergonepeerreview,suchasthroughtheEPAScienceAdvisoryBoard,theNationalAcademies,andotherprocesses.

2.3.PromotingaCultureforRetrospective

AnalysisandIterativePolicymaking

ThesunsetprovisionsforcleanenergyspendingandtaxexpendituresthroughtheInfrastructureInvestmentandJobsActandtheInflationReductionActcreatewindowsofopportunitiesforhowlookingbackatprogramperformancecaninformsubsequentpolicyactions.Likewise,theiterativeapproachtoregulationscreatesnaturalopportunitiesforexpostevaluationofregulatoryperformance.Anumberofregulatoryauthoritiesoperatethroughanupdatingcycle,suchasEPAairqualitystandards,9DepartmentofEnergyapplianceefficiencystandards,10andDepartmentofTransportationfueleconomystandards11Lookingbackatregulatoryperformanceprovidesanopportunitytolearnabouttheefficacyofruledesignandcompliance

strategiesbyregulatedentities,andsignificantlyenhancesknowledgeofregulatoryimpactsrelativetotheprospectiveanalysisdevelopedattherule-writingstage(Greenstone2009,Sunstein2011,Aldy2014a).

Regulatoryagencies’practicewithrespecttoretrospectivereviewofexistingregulations—whichwouldbeanalogoustoacleanenergyprogramevaluationframework—hasyieldedamixedrecord(Harrington2006,Coglianese2013,Aldy2014a,Bull2015,Cropperetal.2017).EveryadministrationdatingbacktotheCarterAdministrationhascalledonregulatoryagenciestoreviewtheirexistingrules,butthefailuretomeaningfullyinstitutionalizeretrospectivereview,buildacultureofsuchreviewwithinagencies,andappropriatemoniestoensuretheresourcesareavailabletoconductsuchreviews,haveunderminedtheeffectivenessofsuchWhiteHousedirectives.Agencieshavereceivedguidanceonhowtoplanforexpostevaluationsofregulationsduringtherule-makingstage,butfewhavemovedforwardwithsuchstrategies(ACUS2014,Aldy2014a,Cropperetal.2018).

8E.O.13990,86FederalRegister7037,January25,2021.

942USC7409(d).

1042USC6313(a)(6(C).

1149USC32902(k)(3).

6

Promotingacultureforretrospectiveanalysisstartswithinstitutionalizingitsusebypoliticalleadersandthepolicyprocess.Ifthereisneitheranobviousaudiencefortheanalysisnoraprocessforusingtheoutputsoftheanalysisforimprovingpolicy,thenagencieswillconsidersuchevaluationsofpoliciesinpracticealowpriority.DuringtheObamaAdministration’sretrospectiverevieweffort,agenciespostedonlinethelistofrulesunderreviewandtheresultsofthosereviews.Overtime,however,theseperiodicupdatesbyregulatoryagenciesreceivedlessattentionfromtheWhiteHouse,stakeholders,andthemedia(Aldy2014a).

3.ConductingProgramEvaluations:LessonsfromAcademicResearchoftheAmericanRecoveryandReinvestmentActof200912

Thechallengeinlearningaboutpolicyimpactsliesinidentifyingtheappropriatedataandimplementingtherigorousevaluationtoolstoproducearobustunderstandingoftheimpactofcleanenergyprograms.Aprogramevaluationismuchmorethansimplyreportingthenumberofparticipatingfirmsorhouseholdsinaprogram,ortakingsuchacountandmultiplyingitbyanengineering-basedoutcome,suchasexpectedenergysavings.Empiricalsocialscientistshavedevelopedanarrayofevaluationtools—fieldexperimentsthatimplementrandomizedcontroltrialsaswellasquasi-experimentalmethodsthatattempttoreplicatethefundamentalcharacteristicsofarandomizedcontroltrial(e.g.,AngristandPischke2008,LeeandLemieux2010,DiNardoandLee2011,ImbensandRubin2015)—toestimatethevariousoutcomescausedbyaprogramorpolicyintervention.

Estimatingthecausalimpactofacleanenergyprogramrequiresinformationaboutboththosewhoparticipateintheprogramandthosewhodonot.Simplycollectingdatafromthosereceivinggrantsorclaimingtaxcreditswouldbeinsufficient;rigorousanalysisalsodependsondataaboutthosehouseholdsandbusinessesthataresimilartothesubsidyrecipientsbutarenotrecipients.Thesenon-participantdataprovidethebasisforthecounterfactual—whatwouldhavehappenedintheabsenceofthepolicy—thatenablesanalysisofprogramperformance.Ineffect,dataonprogramparticipantsrepresentsinformationona“treatment”groupanddataonnon-participantsrepresentstheinformationona“control”group,justasinarandomizedexperimenttoevaluatetheimpactsofadrugorvaccine.

12ForgeneralassessmentsoftheRecoveryAct’scleanenergypackage,refertoAldy

(2013),Carley(2016),andBarbier(2020).

LearningHowtoBuildBackBetterthroughCleanEnergyPolicyEvaluation7

TheambitiousspendingandpolicyexperimentationundertheAmericanRecoveryandReinvestmentActof2009hasbeensubjecttoextensiveprogramevaluationsintheacademicliterature.TheRecoveryActprovidedabout$100billionincleanenergyspendingandtaxexpenditurestopromotedeploymentoflow-carbontechnologiesandspureconomicactivity(Aldy2013,CEA2016).TheenergylandscapehaschangedramaticallysincetheRecoveryActwassignedintolawinFebruary2009:utility-scalesolarpowergenerationismorethan100timesgreaterandwindpowergenerationisnearlyseventimesgreatertodaythanin2008(EIAn.d.).PolicymakerscoulddrawfromthispastexperienceinevaluatingRecoveryActprogramstoapplyprogramevaluationmethodstonewcleanenergypoliciesgoingforward.

Thissectionpresentsillustrationsofmethodsforconductingprogramevaluationsthatcrediblyestimatethecausalimpactsofcleanenergyprograms.IshowhoweachofthesemethodscanbeappliedusingstudiesoffourcleanenergyprogramssupportedbytheRecoveryAct.Ineachcase,Iopenbydescribingthepotentialbiasesthatmayresultinmisleadingclaimsofprogramperformancebasedonprogramparticipationratesandengineeringassumptions.ThenIdescribetheauthors’studyandapplicationofastatisticalmethodthatcanaccountforandminimizethesebiases.Foreachcasestudy,Inotehowthestudy’smethodcouldinformfutureprogramevaluationsforspecificcleanenergyprogramsintheInfrastructureInvestmentandJobsActandthe

InflationReductionAct.

3.1.LearningthroughRandomization:The

WeatherizationAssistanceProgram

The2009RecoveryActprovidednearly$5billionoffundingforWeatherizationAssistancePrograms(WAP)implementedatthestateandlocallevels.Theseweatherizationprogramsfinanceenergy-efficiencyandconservationimprovementsintheresidentialdwellingsofhouseholdswithincomebelowaspecifiedthreshold.

3.1.1.PotentialBiases

TheDepartmentofEnergyhastypicallyestimatedthereducedenergydemandandassociatedenergybillsavingsofweatherizationthroughengineering-basedevaluations(e.g.,OakRidgeNationalLaboratory2015).Engineering-basedanalysessufferfromthreepotentialshortcomings.First,theweatherizationinvestmentinpracticemayyielddifferentenergysavingsbecauseofsimplifyingassumptionsintheengineeringmodelorvariationsinthequalityofthecontractorsundertakingthework.Second,individualsoptingtoparticipateinaweatherizationprogrammaybefundamentallydifferent—perhapstheyaremoreenergyorenvironmentallyconscious—fromthegeneralpopulation,andtheirbehaviormaynotberepresentative.Finally,weatherizationlowersthecostofanenergyservice—suchasheatingahometoagiventemperature.Residentsofaweatherizedhomemayadjustthethermostat,orbuymoreenergy-consumingappliances,andthisso-called“reboundeffect”wouldoffsetsomeoftheenergysavings.

8

3.1.2.AnEvaluationStrategytoAddresstheBiases

Inpolicydebates,therehasoccasionallybeenatensionbetweenadvocatesofprogramevaluation—whoargueforimplementingapublicprogramthrougharandomizedcontroltrialtoenablerigorousassessment—andagencystafforpoliticianswhoclaimthattheprogramshouldbeavailabletoeveryonewhoiseligible.Fowlieetal.(2018)developedacleverwayofresolvingthistension.WorkingwithalocalweatherizationprograminMichigan,theydevelopedarandomizedencouragementprogram—theydidnotalterwhowaseligibleforthismeans-testedprogram,buttheyrandomizedwhoreceivedinformationandtechnicalassistanceforapplyingforweatherizationaid.Thisrandomizationsatisfiedpoliticalconstraints,andalsoallowedtheresearcherstoensurethattheirresultswerenotconfoundedby,forexample,self-selectionintotheprogrambythosemorelikelytobeenergy-conscious.They