利用大语言模型促进综合图学习能力

#利用大语言模型

促进综合图学习能力02

大语言模型进行图学习的现状概述01

为什么应用大语言模型进行图学习

03

大语言模型促进跨领域跨任务的统一图学习04

潜在研究方向目录#01为什么应用大语言模型进行图学习为什么应用大语言模型进行图学习大语言模型的能力图数据的特征为什么应用大语言模型进行图学习大语言模型的能力LLMs

have

demonstrated

their

strong

text

encoding/decoding

ability.Zhao

W

X,

Zhou

K,

Li

J,

et

al.

A

survey

of

large

language

models[J].

arXiv

preprint

arXiv:2303.18223,2023.为什么应用大语言模型进行图学习大语言模型的能力LLMs

have

shown

newly

found

emergent

ability

(e.g.,

reasoning).Wei

J,

Wang

X,

Schuurmans

D,

et

al.

Chain-of-thought

prompting

elicits

reasoning

in

largelanguage

models[J].

Advances

in

neural

information

processing

systems,

2022,

35:

24824-24837.为什么应用大语言模型进行图学习图数据的特征In

real

world,

text

and

graph

usually

appears

simultaneously.Text

data

are

associated

with

rich

structure

information

in

the

form

of

graphs.Graph

data

are

captioned

with

rich

textual

information.#02大语言模型进行图学习的现状概述大语言模型进行图学习的现状概述不同的图数据应用场景图任务中大语言模型的不同角色不同的图数据应用场景Jin

B,

Liu

G,

Han

C,

et

al.

Large

language

models

on

graphs:

A

comprehensive

survey[J].

arXivpreprintarXiv:2312.02783,

2023.大语言模型进行图学习的现状概述大语言模型进行图学习的现状概述不同的图数据应用场景：Pure

GraphDefinition:

Graph

with

no

text

information

or

no

semantically

rich

text

information.

eg.

traffic

graphsor

power

transmission

graph.Problems

on

Pure

Graphs:graph

reasoning

tasks

likeconnectivityshortestpathsubgraph

matchinglogical

rule

induction…Wang

H,

Feng

S,

He

T,

et

al.

Can

language

models

solve

graph

problems

in

natural

language?[J].Advances

in

Neural

Information

Processing

Systems,

2024,

36.大语言模型进行图学习的现状概述不同的图数据应用场景：Pure

GraphGraph

with

no

text

information

or

no

semantically

rich

text

information.

eg.

traffic

graphs

or

powertransmission

graph.Wang

H,

Feng

S,

He

T,

et

al.

Can

language

models

solve

graph

problems

in

natural

language?[J].Advances

in

Neural

Information

Processing

Systems,

2024,

36.大语言模型进行图学习的现状概述不同的图数据应用场景：Text-Paired

GraphSeidl,

P.,

Vall,

A.,

Hochreiter,

S.,

&

Klambauer,

G.,

Enhancing

activity

prediction

models

in

drug

discoverywith

the

ability

to

understand

human

language,

in

ICML,

2023大语言模型进行图学习的现状概述不同的图数据应用场景：Text-Paired

GraphSeidl,

P.,

Vall,

A.,

Hochreiter,

S.,

&

Klambauer,

G.,

Enhancing

activity

prediction

models

in

drug

discoverywith

the

ability

to

understand

human

language,

in

ICML,

2023大语言模型进行图学习的现状概述不同的图数据应用场景：Text-Attributed

GraphRuosong

Ye,

Caiqi

Zhang,

Runhui

Wang,

Shuyuan

Xu,

and

Yongfeng

Zhang.

2024.

Language

is

All

a

Graph

Needs.

In

Findings

of

the

Association

for

Computational

Linguistics:

EACL

2024,

pages

1955–1973,

St.

Julian’s,

Malta.

Association

for

Computational

Linguistics.大语言模型进行图学习的现状概述不同的图数据应用场景：Text-Attributed

GraphRuosong

Ye,

Caiqi

Zhang,

Runhui

Wang,

Shuyuan

Xu,

and

Yongfeng

Zhang.

2024.

Language

is

All

a

Graph

Needs.

In

Findings

of

the

Association

for

Computational

Linguistics:

EACL

2024,

pages

1955–1973,

St.

Julian’s,

Malta.

Association

for

Computational

Linguistics.大语言模型进行图学习的现状概述图任务中大语言模型的不同角色LLM

as

Enhancer/EncoderLLM

as

PredictorLLM

as

Aligner大语言模型进行图学习的现状概述Eembedding-based图任务中大语言模型的不同角色:LLM

as

Enhancer/EncoderExplanation-based大语言模型进行图学习的现状概述LLM

as

Enhancer/Encoder:

Explanation-basedICLR’24Basically,

using

T

andA,to

generate

P

and

E,then

use

T,

A,

P,

Easenriched

text

feature.He

X,

Bresson

X,

Laurent

T,

et

al.

Explanations

as

features:

Llm-based

features

for

text-attributed

graphs[J].arXiv

preprint

arXiv:2305.19523,

2023.大语言模型进行图学习的现状概述LLM

as

Enhancer/Encoder:

Explanation-basedHe

X,

Bresson

X,

Laurent

T,

et

al.

Explanations

as

features:

Llm-based

features

for

text-attributed

graphs[J].arXiv

preprint

arXiv:2305.19523,

2023.大语言模型进行图学习的现状概述LLM

as

Enhancer/Encoder:

Embedding-basedObservation:Fine-tune-based

LLMsmay

fail

at

low

labelingrate

settings.Chen,

Z.,

Mao,

H.,

Li,

H.,

Jin,

W.,

Wen,

H.,

Wei,

X.,

Wang,

S.,

Yin,

D.,

Fan,

W.,

Liu,

H.,

&

Tang,

J.

(2024).

Exploring

thePotential

of

Large

Language

Models

(LLMs)

in

Learning

on

Graphs

(arXiv:2307.03393).

arXiv./abs/2307.03393Low

label

ratioHigh

label

ratio大语言模型进行图学习的现状概述LLM

as

Enhancer/Encoder:

Embedding-basedObservation:Under

embedding-basedstructure,

the

combination

ofdeep

sentence

embedding

withGNNs

makes

a

strong

baseline.Low

label

ratioChen,

Z.,

Mao,

H.,

Li,

H.,

Jin,

W.,

Wen,

H.,

Wei,

X.,

Wang,

S.,

Yin,

D.,

Fan,

W.,

Liu,

H.,

&

Tang,

J.

(2024).

Exploring

thePotential

of

Large

Language

Models

(LLMs)

in

Learning

on

Graphs

(arXiv:2307.03393).

arXiv./abs/2307.03393High

label

ratio大语言模型进行图学习的现状概述图任务中大语言模型的不同角色:LLM

as

PredictorFlatten-basedGNN-based大语言模型进行图学习的现状概述LLM

asPredictor:

Flatten-basedGuo,

J.,

Du,

L.,

&

Liu,

H.

(2023).

Gpt4graph:

Can

large

language

models

understand

graphstructured

data?

an

empirical

evaluation

and

benchmarking.

arXiv

preprint

arXiv:2305.15066.大语言模型进行图学习的现状概述LLM

asPredictor:

Flatten-basedGuo,

J.,

Du,

L.,

&

Liu,

H.

(2023).

Gpt4graph:

Can

large

language

models

understand

graphstructured

data?

an

empirical

evaluation

and

benchmarking.

arXiv

preprint

arXiv:2305.15066.大语言模型进行图学习的现状概述LLM

as

Predictor:

GNN-basedTang,

Jiabin,

et

al.

"Graphgpt:

Graph

instruction

tuning

for

large

language

models."

arXiv

preprintarXiv:2310.13023

(2023).大语言模型进行图学习的现状概述LLM

as

Predictor:

GNN-basedTang,

Jiabin,

et

al.

"Graphgpt:

Graph

instruction

tuning

for

large

language

models."

arXiv

preprintarXiv:2310.13023

(2023).大语言模型进行图学习的现状概述图任务中大语言模型的不同角色:LLM

as

Aligner大语言模型进行图学习的现状概述LLM

as

Aligner:ContrastiveWen,

Z.,

&

Fang,

Y.

(2023).

Prompt

tuning

on

graph-augmented

low-resource

textclassification.

arXiv

preprint

arXiv:2307.10230.大语言模型进行图学习的现状概述LLM

as

Aligner:DistillationMavromatis,

Costas,

et

al.

"Train

your

own

gnn

teacher:

Graph-aware

distillation

on

textualgraphs."

Joint

European

Conference

on

Machine

Learning

and

Knowledge

Discovery

in

Databases.Cham:

Springer

Nature

Switzerland,

2023.#03大语言模型促进跨领域跨任务的统一图学习大语言模型促进跨领域跨任务的统一图学习“Cross

Domain”

before

LLMsCross

Domain

Graph

Learning

with

LLM大语言模型促进跨领域跨任务的统一图学习“Cross

Domain”

before

LLMsKDD’20:

“We

design

Graph

Contrastive

CodingQiu,

Jiezhong,

et

al.

"Gcc:

Graph

contrastive

coding

for

graph

neural

network

pre-training."

Proceedings

of

the

26th

ACM

SIGKDD

international

conference

on

knowledge

discovery&

data

mining.

2020.(GCC)—a

self-supervised

graph

neural

network

pre-training

framework—to

capture

the

universal

networktopological

properties

across

multiple

networks.”Limitation:

the

node

features

are

notthe

same,

among

graphs

fromdifferent

domain.大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMOne

for

all:Towards

trainingone

graphmodel

for

allclassificationtasksLiu,

Hao,

et

al.

"One

for

all:

Towards

training

one

graph

model

for

all

classification

tasks."

arXivpreprint

arXiv:2310.00149

(2023).大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMLiu,

Hao,

et

al.

"One

for

all:

Towards

training

one

graph

model

for

all

classification

tasks."

arXivpreprint

arXiv:2310.00149

(2023).大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMOFA

successfully

enableda

single

graph

model

to

be

effective

onall

graph

datasets

across

differentdomains

as

OFA-joint

performs

well

onall

datasets.Further,

we

can

see

that

OFA-jointachieves

better

results

on

most

of

thedatasets

compared

toOFA-ind.

Thismay

indicate

that

by

leveraging

thetext

feature,

the

knowledge

learnedfrom

one

domain

can

be

useful

for

thelearning

of

other

domains.Liu,

Hao,

et

al.

"One

for

all:

Towards

training

one

graph

model

for

all

classification

tasks."

arXivpreprint

arXiv:2310.00149

(2023).大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMOverview

of

UniGraph

framework.

In

pre-training,

we

employ

a

self-supervised

approach,

leveragingTAGs

to

unify

diverse

graph

data.

This

phase

involves

a

cascaded

architecture

combining

LMs

andGNNs.He,

Yufei,

and

Bryan

Hooi.

"UniGraph:

Learning

a

Cross-Domain

Graph

Foundation

Model

FromNatural

Language."

arXiv

preprint

arXiv:2402.13630

(2024).大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMWe

can

observe

thatpre-training

ongraphs

from

the

same

domain

enhancesthe

performance

of

downstream

tasks.This

suggests

that

in-domain

transferremains

simpler

than

cross-domaintransfer.He,

Yufei,

and

Bryan

Hooi.

"UniGraph:

Learning

a

Cross-Domain

Graph

Foundation

Model

FromNatural

Language."

arXiv

preprint

arXiv:2402.13630

(2024).Experiment

results

in

few-shot

transfer.大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMTan,

Yanchao,

et

al.

"MuseGraph:

Graph-oriented

Instruction

Tuning

of

Large

Language

Modelsfor

Generic

Graph

Mining."

arXiv

preprint

arXiv:2403.04780

(2024).大语言模型促进跨领域跨任务的统一图学习Cross

Domain

Graph

Learning

with

LLMTan,

Yanchao,

et

al.

"MuseGraph:

Graph-oriented

Instruction

Tuning

of

Large

Language

Modelsfor

Generic

Graph

Mining."

arXiv

preprint

arXiv:2403.04780

(2024).#04潜在研究方向潜在研究方向What

LLMs

truly

learned

from

GraphsObservation

1:LLMs

interpret

inputs

more

as

contextual

paragraphs

than

as

graphs

with

topologicalstructures.

Neither

linearizing

nor

rewiring

ego-graph

has

significant

impact

on

theclassification

performance

of

LLMs.Linearize

ego-graph:

We

create

a

linearized

version

of

the

graph-structured

prompts

by

onlykeeping

all

neighbors’

text

attributes

in

the

prompts.Huang,

Jin,

et

al.

"Can

llms

effectively

leverage

graph

structural

information:

when

and

why."

arXivpreprint

arXiv:2309.16595

(2023).潜在研究方向What

LLMs

truly

learned

from

GraphsObservation

1:LLMs

interpret

inputs

more

as

contextual

paragraphs

than

as

graphs

with

topologicalstructures.

Neither

linearizing

nor

rewiring

ego-graph

has

significant

impact

on

theclassification

performance

of

LLMs.Rewire

ego-graph:We

randomly

rewire

theego-graph

by

differentstrategies.

Then

wecompare

theperformance

of

MPNNsand

LLMs

under

eachstrategy.Huang,

Jin,

et

al.

"Can

llms

effectively

leverage

graph

structural

information:

when

and

why."

arXivpreprint

arXiv:2309.16595

(2023).潜在研究方向What

LLMs

truly

learned

from

GraphsObservation

2:LLMs

benefit

from

structural

information

only

when

the

neighborhood

is

homophilous,which

means

the

neighbors

contain

phrases

related

to

the

groundtruth

label

of

the

targetnode.Huang,

Jin,

et

al.

"Can

llms

effectively

leverage

graph

structural

information:

when

and

why."

arXivpreprint

arXiv:2309.16595

(2023).潜在研究方向What

LLMs

truly

learned

from

GraphsObservation

3:LLMs

benefit

from

structural

information

when

the

target

node

does

not

contain

enoughphrases

for

the

model

to

make

reasonable

prediction.Huang,

Jin,

et

al.

"Can

llms

effectively

leverage

graph

structural

information:

when

and

why."

arXivpreprint

arXiv:2309.16595

(2023).潜在研究方向Truly

“Generative”

Cross

Domain

LLM-based

Graph

LearningIs

there

universal

structure

features

that

benefit

for

graph

learningof

graph

from

different

domain?How

can

these

complex

topological

features,

instead

of

the

textcontext,

be

really

captured

by

LLMs?致谢Large

Language

Models

on

Graphs:

A

Comprehensive

Survey致谢言鹏韦浙江大学信息资源管理系2021级博士研究生阿里巴巴通义实验室实习生ReferencesZhao

W

X,

Zhou

K,

Li

J,

et

al.

A

survey

of

large

language

models[J].

arXiv

preprint

arXiv:2303.18223,

2023.Huang,

Jin,

et

al.

"Can

llms

effectively

leverage

graph

structural

information:

when

and

why."

arXiv

preprintarXiv:2309.16595

(2023).Tan,

Yanchao,

et

al.

"MuseGraph:

Graph-oriented

Instruction

Tuning

of

Large

Language

Models

for

Generic

GraphMining."

arXiv

preprint

arXiv:2403.04780(2024).He,

Yufei,

and

Bryan

Hooi.

"UniGraph:

Learning

a

Cross-Domain

Graph

Foundation

Model

From

Natural

Language."

arXivpreprint

arXiv:2402.13630

(2024).Liu,

Hao,

et

al.

"One

for

all:

Towards

training

one

graph

model

for

all

classification

tasks."

arXiv

preprintarXiv:2310.00149

(2023).Qiu,

Jiezhong,

et

al.

"Gcc:

Graph

contrastive

coding

for

graph

neural

network

pre-training."

Proceedings

of

the

26th

ACMSIGKDD

international

conference

on

knowledge

discovery

&

data

mining.

2020.Mavromatis,

Costas,

et

al.

"Train

your

own

gnn

teacher:

Graph-aware

distillation

on

textual

graphs."

Joint

European

Conferenceon

Machine

Learning

and

Knowledge

Discovery

in

Databases.

Cham:

Springer

Nature

Switzerland,

2023.Wen,

Z.,

&

Fang,

Y.

(2023).

Prompt

tuning

on

graph-augmented

low-resource

text

classification.

arXiv

preprintarXiv:2307.10230.Tang,

Jiabin,

et

al.

"Graphgpt:

Graph

instruction

tuning

for

large

language

models."

arXiv

preprint

arXiv:2310.13023

(2023).Guo,

J.,

Du,

L.,

&

Liu,

H.

(2023).

Gpt4graph:

Can

large

language

models

understand

graph

structured

data?

an

empiricalevaluation

and

benchmarking.

arXiv

preprint

arXiv:2305.15066.ReferencesXie,

Han,

et

al.

"Graph-aware

language

model

pre-training

on

a

large

graph

corpus

can

help

multiple

graphapplications."

Pro