机器智能体的混合认知模型 A HYBRID COGNITIVE MODEL FOR MACHINE AGENTS IN PROJECT AND ACTION TEAMS

AHYBRIDCOGNITIVEMODELFOR

MACHINEAGENTSINPROJECTAND

ACTIONTEAMS

DISSERTATION

JoshuaA.Lapso,Major,USAFAFIT-ENG-DS-23-D-031

DEPARTMENTOFTHEAIRFORCEAIRUNIVERSITY

AIRFORCEINSTITUTEOFTECHNOLOGY

Wright-PattersonAirForceBase,Ohio

DISTRIBUTIONSTATEMENTA

APPROVEDFORPUBLICRELEASE;DISTRIBUTIONUNLIMITED.

TheviewsexpressedinthisdocumentarethoseoftheauthoranddonotreflecttheofficialpolicyorpositionoftheUnitedStatesAirForce,theUnitedStatesDepartmentofDefenseortheUnitedStatesGovernment.ThismaterialisdeclaredaworkoftheU.S.GovernmentandisnotsubjecttocopyrightprotectionintheUnitedStates.

AFIT-ENG-DS-23-D-031

AHYBRIDCOGNITIVEMODELFOR

MACHINEAGENTSINPROJECTANDACTIONTEAMS

DISSERTATION

PresentedtotheFaculty

GraduateSchoolofEngineeringandManagement

AirForceInstituteofTechnology

AirUniversity

AirEducationandTrainingCommand

inPartialFulfillmentoftheRequirementsforthe

DegreeofDoctorofPhilosophyinComputerEngineering

JoshuaA.Lapso,B.S.C.E.,M.S.C.O.Major,USAF

21December2023

DISTRIBUTIONSTATEMENTA

APPROVEDFORPUBLICRELEASE;DISTRIBUTIONUNLIMITED.

AFIT-ENG-DS-23-D-031

AHYBRIDCOGNITIVEMODELFOR

MACHINEAGENTSINPROJECTANDACTIONTEAMS

DISSERTATION

JoshuaA.Lapso,B.S.C.E.,M.S.C.O.

Major,USAF

CommitteeMembership:

Dr.GilbertL.PetersonChairman

Dr.MichaelE.MillerMember

Dr.DouglasD.HodsonMember

Dr.LarryW.BurggrafDean’sRepresentative

iv

AFIT-ENG-DS-23-D-031

Abstract

Theinteractionsbetweenhumansandmachineagentsareubiquitousandperme-atetheconceptualboundariesoforganizedgroupsandactivities.Highperforminghumanteamstranscendcomplexdomainuncertaintybyachievinganemergentstateofsharedcognition,inwhichknowledgeisorganized,represented,anddistributedtoteammembersforrapidexecution.However,thisrequiresthatindividualsemitperceivablequalitiesuponwhichothermemberscanmakeinferencesaboutintent.Inpursuitoffuturehumanandmachineteamstudies,thisresearchpresentsahy-bridcognitivemodelformachineagentsinfullycooperativeandsemi-cooperativeactionandprojectteams.Thehybridcognitivemodelunifiesthecharacteristicsofthesharedmentalmodelandtransactivememorysystem.Theresultantmodelfacilitatesanytimeselectionoverthetwocognitiverepresentationswiththecomputa-tionalcomplexityofasinglemodel.Evaluationofthehybridcognitivemodeloccursinmulti-agentdomainswithincreasingcomplexityandlevelsofcooperation.Agentper-formanceisassessedaccordingtofourcognitivecharacteristicsthatcaptureaspectsofthenaturesandformsofcognitionfoundinprojectandactionteams.Thestudiesutilizeamixedmethodsapproachintheanalysisoffourestablishedcharacteristicsandmeasures.Theresultsdemonstratethatagentsusingthecognitivemodelformalignedrepresentationsthatencodestructural,perceptual,andinterpretivecognitiveforms.Additionally,theresultssuggestthatagentsemployingthehybridcognitivemodelcanswitchbetweencompositionalandcompilationalnaturesofemergenceasnecessarytointegratebehaviorsorknowledge.

v

Acknowledgements

Tomyresearchadvisorandmentor

Thankyouforinvestingyourtimeinme.Thankyouforhelpingmegrowasascientist.Thankyouforpatientlyguidingmethroughthisacademicendeavorandsupportingmeamidthepersonalchallenges.

Tomycommitteemembers

Thankyouforalltheinvaluableinsightsandfeedback.YoumadeanincredibleimpactonmeandIlookforwardtofuturecollaborations.

Tothedepartmentleadership

Thankyouforencouragingmethroughoutthisjourneyandkeepingmeoptimistic.I’mproudtobeamemberofthisfacultyteam.

Tomywife

Mahalagakongasawa,thankyouformakingthisdreampossible.Thankyouforsacrificingtimeandenergytokeepmegoing.Thankyouforpickinguptheslackandbeingthebestmemberofourteam.

Tomykids

Thankyouforinspiringme.Thankyouforbelievinginme.Thankyouforremindingmetotakebreaksandshareinyourjoy.Papalovesyou!

JoshuaA.Lapso

vi

TableofContents

Page

Abstract iv

Acknowledgements v

ListofFigures viii

ListofTables x

I.Introduction 1

1.1ProposedResearch 4

1.2AssumptionsandLimitations 5

1.3ModelImplementation 6

1.4EvaluationMeasures 8

1.5SummaryofRemainingSections 11

II.RelatedWork 12

2.1Teams 12

2.2SharedCognition 14

2.3CausalityandBayesianModels 21

2.4MachineAgentModelsandGameTheory 22

2.5ReinforcementLearning 26

2.6Summary 27

III.TheHybridCognitiveModelforMachineAgents:

Formalization,Generalization,andProblemDomains 28

3.1CognitiveTeamModels 28

Teams 29

CognitiveModelsforAdviceTeams 30

CognitiveModelsforProductionTeams 33

CognitiveModelsforProjectandActionTeams 35

3.2Formalization 36

3.3GeneralizedGraphicalModelsandAlgorithmicForms 38

3.4ProblemDomains 44

Triage 44

IncidentResponse 45

Diplomacy 47

vii

Page

IV.StudyingHCMPerformanceinCollaborativeProjectTeams 49

4.1Introduction 50

4.2SpecializedInstantiationforCollaborativeProject

Teams 52

4.3PreliminaryEvaluation:FullyFactoredBeliefs 57

4.4Experimentation 59

4.5Results 62

Characteristic1(C1) 62

Characteristic2(C2) 63

Characteristic3(C3) 64

4.6Conclusion 66

V.CognitiveTeamModelsinPressDiplomacy 67

5.1Introduction 67

5.2AHybridCognitiveModelforPressDiplomacy 70

ModelFormulation 71

GraphicalModel 73

ModelElements 73

AlgorithmicApproach 78

5.3DiplomacyVignettes 84

PostureGameStage 84

EarlyGameStage 88

MiddleGameStage 91

5.4Results 95

ParameterTuning 95

Characteristic4(C4) 97

5.5Summary 99

VI.ConclusionandFutureWork 101

6.1SummaryofContributions 101

RetrospectiononFormalizingtheHCM 102

6.2FutureWork 105

Bibliography 107

viii

ListofFigures

FigurePage

1Acomparisonofthenatureofemergence 15

2Acomparisonofsharedmentalalignment 18

3Ademonstrationoftransactiveencoding 20

4AgeneralizedDBM 22

5Partiallyobservablestochasticgames 23

6Individualencoding 31

7Transactiveinterpretation 32

8Transactiveencoding 33

9Individualandsharedmentalmodels 34

10AvisualizationoftheHCMforthreeteammembers 36

11Acomparisonofactiveelementsinthehybridcognitive

model 37

12TheHCMasaDBNfortwoagents 39

13AflowchartforatwoHCMagents 40

14TheHCMasaDBNforfullyobservabledomains 42

15AtwoagentvisualizationoftheTriageproblemdomain 44

16AnabstractdiscretizationoftheIncidentResponse

domain 46

17TheDiplomacyGameBoard[1] 47

18TheHCMasaDBNformultipleagents 55

19Averagecumulativerewards 58

20Beliefstatetrackingaccuracy 59

21AnabstractdiscretizationoftheIncidentResponse

domain 60

ix

FigurePage

22Cumulativemessageexchanges 64

23Internalstatetrackingaccuracy 65

24TheHCMasaDBNforPressDiplomacy 74

25Theinternalcognitivestatetransitions 74

26AnorientationalviewofaDiplomacyPower 77

27Anorientationalviewofeachpowernodeconnectedby

weightededges 77

28Austria’scognitiverepresentationatthebeginningof

play 85

29Austria’scognitiverepresentationafterthefirst

Diplomaticphase 87

30Germany’spriorcognitiverepresentationintheearly

gamestage 89

31Germany’sposteriorcognitiverepresentationinthe

earlygamestage 90

32Turkey’spriorcognitiverepresentationinthemiddle

gamestage 93

33Turkey’sposteriorcognitiverepresentationinthe

middlegamestage 94

34RMSerrorforparticlecountinsampledvariables 96

35RMSerrorforDutchtracewithingamestages 97

36PointswonbyeachpowerplayedbyanHCMagent

over840games 99

37Cumulativepointswonbyeachpowerplayedbyan

HCMagentatdiscreteintervalsthroughoutthe120

repeatedgames 100

x

ListofTables

TablePage

1Acomparisonofabstractteamtypes 13

2CharacteristicsofSMMsandTMSs[2] 17

3Domaincomplexityforvariousagentconfigurations 63

4HCMWeightingElements 76

5Errormatrixbetweenprojectedandactualordersets

duringtheposturestage 87

6Theerrormatrixbetweenprojectedandactualorder

setsduringtheearlystage 91

7Theerrormatrixbetweenprojectedandactualorder

setsduringthemiddlestage 95

1

AHYBRIDCOGNITIVEMODELFOR

MACHINEAGENTSINPROJECTANDACTIONTEAMS

I.Introduction

Studiesinthehumanfactorsandcognitivepsychologydisciplinesprovidecriti-calinsightsintothecharacteristicsofhighperformingteams.Theseteamstranscendcomplexdomainuncertaintybyachievingastateofsharedcognition,inwhichknowl-edgeisorganized,represented,anddistributedtoteammembersforrapidexecution[3].Muchoftheworkinmodelingsharedcognitionleverageseithertheconceptofasharedmentalmodel(SMM)[4]oratransactivememorysystem(TMS)[5,6].SMMsarejointunderstandingsoffunctionsorplanssharedamongteammembers.Alternatively,aTMSoffersteammembersaglobalcontextofinformation,mappingwhereknowledgeresidesandthecredibilityoftheknowledgeholder.

Teamscanbecategorizedintofourtypes:(1)advice,(2)production,(3)project,or(4)actionteamsSundstrom,etal.[7].DeChurchandMesmer-Magnus[3]proposedthreecomparableteamcategories:(1)action,(2)decision-making,and(3)projectteams,wherecharacteristicsofSundstrom,etal.’s[7]actionteamisredistributedintotheiractionandprojectteamdefinitions.ThedifferencebetweenSundstrom,etal.’s[7]projectandactionteamsisthelifespanandtaskenvironment.Successfuladviceteamsintegrateknowledgeandproductionteamsintegratebehaviors.Foractionandprojectteams,performancerestsonateamsabilitytointegratebehaviorsandknowledgesimultaneously.

Eachmodelcapturesvaluableinsightsonhowhumansformsharedcognition.SMMsenablehowalignmentinprocessunderstandingandunifiedexecutioninte-

2

gratesbehaviorsandimprovesperformanceinepisodicphysicaltasks(production).TMSsexhibitthatdiversethoughtanddistributedexpertiseintegratesknowledgeandyieldsbetterdecisionsinsequentialknowledgetasks(advice).Neithermodelisasufficientcognitiverepresentationforteamsthatintegrateknowledgeandbehaviorssimultaneously(projectoraction)[3,8].

Projectandactionteamsoperateinenvironmentswithhighlevelsofuncertainty,looselydefinedboundaries,andvaryingmeasuresofsuccess.Instancesoftheseteamtypesaregenerallyfoundwhereinnovation,transformation,orimprovisationisre-quired[7,9,3].Innovativeteamsoftenperformnovelresearchorprototypicaldesign.Transformativeteamsmayleadlargecorporatemergersoractasfirstresponderstopandemicsornaturaldisasters.Improvisationalteamstypicallymanifestassportingclubs,militaryunits,orcrisisnegotiators.Theenvironmentinwhicheachteamoper-atesplacesuniqueconstraintsontheintegrationofknowledgeandbehaviorsandtheabilitytogeneralizebehaviorandknowledgehasnotbeengeneralizedacrosshumansandmachineagents.

Someconstraints,communicationforinstance,applytohumansandmachineagents.Additionalconstraintsonhumanteammembersmanifestasresourcesre-quiredtoattainphysicalskillsorcognitiveproficiencies.Alternatively,constraintsonmachineagentsarerealizedasdesigncomplexitiesindomainrepresentations.Di-rectlycomputingoptimalsolutionsforinterdependentbehaviorsincomplexdomainrepresentationsbecomesintractableforasinglemachineagent,butwellestablishedapproximationmethodsyieldnear-optimalsolutions.Similarlycomplexdomainswithmultiplemachineagentsrequireomnisciencetoleveragethesameapproximationmethods.Omnisciencecanbesimulatedthroughbroadcastcommunicationbetweenallothermachineagents[10].However,thecommunicationoverheadthenbecomesalimitingfactorasagentsmustcommunicatepriortotakingactionsandafterre-

3

ceivingevidenceaboutchangesinthedomain.Ultimately,acognitiverepresentationforinformationorknowledgenecessarytosupportprojectteamsmustconsidertheintegrationtype,theagentconstraints,andthecollaborativeoradversarialnatureofthedomain.

Competitionexistsindynamicdomainsandthenatureofthiscompetitionrangesfromcollaborativetoadversarial.Humansascribealevelofconfidencetoallpartic-ipantsinthesedomainsthatisagnostictothenatureofcompetition[11,3,12,13].Inmorecollaborativedomainsthislevelofconfidencecanindicateameasureofthor-oughness,expertise,orreliability.Alternatively,thismeasuremaymanifestasaleveloftrustworthinessorcommitment.Asimilarmeasureexistsformachineagentsandispredominatelyevidencedinmulti-agent,generalsumgamesasaprobabilisticdis-tributionoverthelikelihoodthatanotherplayerwillactcountertopersonalinterests[14,15,16].Domaincomplexityincreasesforhuman[17]andmachineagents[18]alike,whenactorsmustcontinuallyinferthenatureofcompetitionandthelevelofconfidence.

Unifyingprominenttheoriesofsharedhumancognitionpresentsanopportunityforreducingthecognitivedemandsofdomaincomplexityformachineagentsthatparticipateinprojectandactionteams[8,3].TheintuitionhereisthatbyleveragingthecharacteristicsofSMMsandTMSs,withinaunifiedmodel,actorsmaybettersimultaneouslyintegrateknowledgeandbehaviors.Thecorollaryformachineagentsisthataunifiedmodelreducestheircomputationalcomplexity.ThisresearchpresentsaHybridCognitiveModel(HCM)thatenhancesmachineagentperformanceinprojectandactionteams.

4

1.1ProposedResearch

Theprimaryresearchquestionis:Canahybridcognitivemodelthatleveragescharacteristicsofbothsharedmentalmodelsandtransactivememorysystemssignif-icantlyimprovetheperformanceofprojectandactionteamsthatincludemachineagents?Thisquestionpartitionsintofoursubquestionsforindependenttestingandanalysisusingqualitativeandquantitativemethods.

Question1:Canmachineagentsthatutilizethehybridcognitivemodelcreateandmaintainfullyfactoredbeliefstates?MachineagentscannotaccessaMarko-viansignalduringexecutioninDecentralizedPartiallyObservableMarkovDecisionProcesses(Dec-POMDPs)becausethemodeldynamicsarespecifiedinjointactionsandjointobservations.MachineagentsinDec-POMDPsleveragethesedynamicstocompresstheentiretyofthemodel’sstatehistoryintoonesufficientforpredictingfuturestates.WehypothesizethatmachineagentscanleveragethecontentsofthehybridcognitivemodeltoconstructasyntheticMarkoviansignal.

Question2:Doesthehybridcognitivemodelreducethecomputationalcom-plexityformulti-agentdecisionproblems?ThecomplexityclassforDec-POMDPbasedmulti-agentsystemsisnondeterministicexponentialtime(NEXP),whiletheirsingleagentequivalentsarepolynomialSPACE(PSPACE-Hard).Wehypothesizethatthehybridcognitivemodelallowsagentstooperateunderintentprojectionsandindividualobservations,reducingthepolicyspacebyanexponentialmultiplier.

Question3:Doesthehybridcognitivemodelreducethecommunicationsover-headrequiredformachineagentsinmulti-agentdecisionproblems,whilemaintainingperformanceneartheperformanceofomniscentagents?IntraditionalDec-POMDPs,machineagentsmustknowwhatactionmembersintendtotakeandtheirsubsequentobservationsoverthenewstate,inordertoformaMarkoviansignal.Ingeneral,thismeansthatforeachdecisions,twomessagesareexchangedbetweenallteam

5

members.Wehypothesizethatthehybridcognitivemodelreducescommunicationcostbecausetheseagentsformstructuralandperceptualcognitivemodelsandonlyneedtocommunicate:(1)agentsexperienceasensoranomalyor(2)theprobabilitydistributionoverthestatespacedevolves.Followingamessageexchange,theagentsretrospectivelyconstructanewbeliefdistribution.

Question4:Isthehybridcognitivemodeleffectiveinnon-cooperativeteams?Humanteamshavefullyalignedgoals.Oftenindividualgoals,biases,andulteriormotivesbecomedetrimentaltoateam.Realignmentofteamgoalsrequirenegoti-ationsandreasoningovermutualbenefits.Thereasoningelementofnegotiationsevokesasenseofcausality.Wehypothesizethatthehybridcognitivemodel,whenunifiedwithafunctionalcausalmodel,willfacilitateaninterpretivecognitionmodelinmachineagents.

1.2AssumptionsandLimitations

Thisworkpursuesanovelhybridcognitivemodelformachineagentsthat:(1)isoptimizedforhumanandmachineteams,(2)generalizeswelltodeterministicandstochasticproblemdomains,and(3)constructsanexplainablethreadforeachde-cision.Assumptionsandlimitationsaremadetoconstraintheproblemforstudy.Itisexpectedthatassumptionswillberelaxedoverthecourseoffollow-onresearchefforts.Thefollowingassumptionsapplytotheresearch:

Assumption1:Trialsareconductedwithinsimulatedenvironmentsandgames

Assumption2:Simulateddomainsincludesinglelocation,agridofdiscretizedstates,broadlydiscretizedstates,andasymmetricdiscretizedgames

Assumption3:Cooperativeagentsarefullycooperativeandwithouthiddenagendas

6

Assumption4:Non-cooperativeagentsrangefrombenevolentadversariestodiamet-ricallyopposedadversaries

Assumption5:Wellestablishednavigationmethodsareleveragedtomarginalizeoutextraneousvariables

Measurestoboundtheproblemsunderstudyareacknowledgedinthefollowinglimitations.

Limitation1:Closeddomainrepresentationsareutilized

Limitation2:AgentsemploylimitedalphabetsandlanguagesforcommunicationLimitation3:Gametreescontainaterminalstate

Limitation4:ElementsoftheHCMareinitiallyuniformorspecifiedaprioribasedoncharacteristicsrelevanttoeachdomain

Limitation5:Thecostofcommunicationisfreeandtheexchangeisinstantaneous

1.3ModelImplementation

TheHybridCognitiveModel’s(HCM)specificationplacesemphasisoncapturingthekeyattributesoftheSMMandTMS,definingthestagesoflearning,andminimiz-ingthemaintenanceoverheadoftheinstantiatedmodel.Keyattributesconsideredforthisresearchwerethenature,form,andcontentofcognitionwithinSMMsandTMSs[3].WedrewinspirationfromhumanmemoryprocessesanddefinedtheHCM’sstagesoflearningasencoding,storage,andretrieval[19].AnovelqualityoftheHCMisthatitsubsumestheSMMintheTMS,composingaunifiedrepresentationwiththemaintenanceoverhead(i.e.,thecognitiveload)ofasinglemodel.TheHCMisimplementedinthreephases:formalization,generalization,andspecialization.

7

FormalizingtheHCMbeganwithananalysisofSMMandTMScharacteristics,followedbycategorizationoffeaturesintoatomicelements,andanexaminationofapplicablealgorithms.TheanalysisofSMMandTMSliteratureunderscoredsalientcharacteristicsofeachmodel’snature,form,andcontentofcognition.Natureofcog-nitiondescribesthedifferencesinmanifestationsfortheindividualversestheteam.Thecognitiveformcapturesthestructural,perceptual,andinterpretiveelementsrep-resentedinthecognition[3].TheoverlapwassparseconsideringthenatureandformofSMMsandTMSs,butthecontentsharedmanyfeatures.ElevenatomiccharacteristicsextractedduringthisanalysiswerecoalescedintheHCM’sformaliza-tion.Additionally,examinationfromacomputationallensidentifiedfourfamiliesofalgorithmsthatdemonstratedtheversatilequalitiesrequiredbytheHCM.

ThegeneralizedalgorithmicexpressionthatemergedfromtheHCM’sformaliza-tionincorporatesfourqualities:(1)Markovianinnature,(2)acausalstructure,(3)learnswithinatransactivecontext,and(4)accommodatesvariablelengthproblemhorizons.Markovmodelscompressamodel’sentirestatehistoryintothecurrentstateandhasrigorousmathematicalfoundation[20].BayesianmodelsemploytheMarkovassumptionandcanfacilitatecausalinference[21]necessaryforinterpre-tiveemergence[3].ReinforcementlearningiscompatiblewithBayesianmodelsandpresentsanadvantageousapproachtoreplicatingthenature,form,andcontentofbothSMMsandTMSs.Lastly,MonteCarlomethodsarewellsuitedforthedynamicrangeofstochasticity,uncertainty,andcomplexityofthedomainsunderstudy—es-peciallythosewithvariablelengthorinfinitesearchhorizons.Thequalitiesexhibitedbythesefourfamiliesofalgorithmsallowawiderangeofmulti-agentproblemsthatrequirespecializedinstantiations.

Graphicalmodelsarespecifiedforeachproblemdomainstudiedbasedon:(1)levelofuncertainty,(2)structuralcomplexity,(3)learningrequirements,and(4)

8

branchingfactorofthesearchspace.WeutilizeseveralMarkovianmulti-agentdeci-sionmodelsfromthefamilyofpartiallyobservablestochasticgames[22]toeffectivelyadaptagentstovariousdomainrepresentationswithoutsacrificingtheHCM’sgener-alizability.Multi-agentdecisionproblemshavenotoriouscomplexitiesthatincreaseexponentiallywithvariableelementsofeachdomain.DynamicBayesianNetworks(DBNs)provideagraphicalrepresentationofMarkovianmodelsthatcanbefac-toredagent-wisewhilemaintainingthecausalpropertiesrequiredbytheHCM[23].Additionally,DBNsarecompatiblewithsample-basedapproximationalgorithmstoovercomethecomplexityofdomainsunderstudyaswellasmodel-freereinforcementlearningalgorithms[24].Temporaldifference(TD)learningisamodel-freefam-ilyofalgorithmsthatareconsonantwithDBNsandsample-basedapproximations[25].Furthermore,thesealgorithmsarebiologicallyinspired,transactiveinnature,andtailorlearningupdatestoonlineandofflineenvironmentsalike[25].AspectsofMonteCarlomethodsarepresentinsample-basedapproximationsandTDlearning.However,thepredominantroleofMonteCarlomethodsintheHCM’sspecializedinstantiationsissearchingvastconstructsforoptimalsolutionsandinformingagentdecisions.

1.4EvaluationMeasures

TheHCM’sefficacywasassessedatthreedistinctmilestones.EachmilestonesignifiedafoundationalcapabilityofmachineagentsutilizingtheHCMandbecameaprerequisiteforsubsequentmilestones.Evaluationmeasureswererecordedateachmilestoneindomainswithincreasingcomplexity.ThefirstmilestoneconcernedanHCMmachineagent’sabilitytoindependentlyreasonaboutanuncertainenviron-ment,whileminimizingunnecessarycommunication.Milestonetwofocusedontheabilityofmachineagentteamstosuccessfullymakedecisionsandperformtasksin

9

purelycollaborativedomains.ThelastmilestoneestablishedthatHCMagentsformandmaintainacausalthreadthataidsexplainabilityovereachagent’sdecisionmak-ingprocess.

ThecognitivecharacteristicsthatestablishedtheHCM’sdesignrequirementsare(C1)balancedcognitiveloadsforeachmember,(C2)efficientcommunicationsbe-tweenmembers,and(C3)independentreasoningabouttheenvironmentandallothermembersinacollaborativesetting,and(C4)causalreasoningaboutothermembersinanon-cooperativesetting.Experimentalmeasuresderivedfromeachcharacteris-ticare(M1)computationalcomplexityofthepolicyspace,(M2)inter-agentmessagecount,(M3)hiddenstatetrackingaccuracy,and(M4)intentpredictionaccuracywithimprovementinrepeatedtrials.

Theevaluationmeasuresestablishedformilestone1aretheaveragecumulativerewardsearnedbytwoagentteamsandtheaccuracyofeachagent’sindependentlyformedbeliefsabouttheenvironmentalvariables.Machineagentsutilizedrewardsastheiroptimizationcriterionandin-turnlearnswhichactionyieldsthemaximumrewardandwhichactiontheircounterpartismostlikelytochoose.Theaveragecumulativerewardsfordistinctteamconfigurationarecomparedagainstallotherconfigurations.Ateam’sbeliefsovertheenvironmentalvariablesimpactdecisionmaking.Inaccuratebeliefscanproducesub-optimaldecisions,disruptlearning,anddegradeperceptualcognition.Accuracyofindependentbeliefsaboutavariable’svalueiscomparedagainstitstruevalue.Aswithcumulativerewards,ateam’saveragebe-liefaccuracyiscomparedagainstallotherteamconfigurations.Hypothesis1assertsthatHCMagentsformasyntheticMarkoviansignal(C1).Therefore,thismilestoneisachievedifteam’semployingtheHCMexhibitmeasurementscomparabletoom-niscientagentswithinidenticalconfigurations,whohaveaccesstoatrueMarkoviansignal.

10

Thesecondmilestonerecordsthetimeeachteamtakestomakedecisions,theaveragenumberofmessagesexchangedpertria