网络嵌入和图卷积神经网络技术实践

1、网络嵌入和图卷积神经网络技术实践技术创新，变革未来网络/图数据图是对于数据的一/通用、全面、复杂的表示形式网络无处不在社交网络生物网络金融网络物联网信息网络物流网络为什么网络很重要？我们很少只关心数据本身，而不关心数据之间的关联Reflected by relational subjectsDecided by relational subjectsTargetImage CharacterizationTargetSocial Capital网络数据对机器学习模型 不友好G = ( V, E )LinksTopologyInapplicability of ML methodsNetwork

2、 DataFeature ExtractionPattern DiscoveryPipeline for network analysisNetwork ApplicationsLearnabilityLearning from NetworksNetwork EmbeddingGNNG = ( V, E )G = ( V )Vector SpacegenerateembedEasy to parallelCan apply classical ML methods网络嵌入 (Network Embedding)网络嵌入的目标GoalSupport network inference in v

3、ector spaceReflect network structureMaintain network propertiesBACTransitivityBasic idea: recursive definition of statesA simple example: PageRank图神经网络GNNF. Scarselli, et al. The graph neural network model. IEEE TNN, 2009.定义在图拓扑上的学习框架Main idea: pass messages between pairs of nodes & agglomerateStack

4、ing multiple layers like standard CNNs:State-of-the-art results on node classification图卷积神经网络GCNT. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. ICLR, 2017.图神经网络GNN简史网络嵌入与图神经网络GraphFeatureNetwork EmbeddingInputModelOutputEmbeddingTask resultsFeatureTopolog

5、y to VectorGCNTask resultsFusion of Topology and FeaturesUnsupervised vs. (Semi-)Supervised图卷积网络 v . 网络嵌入In some sense, they are different.Graphs exist in mathematics. (Data Structure)Mathematical structures used to model pairwise relations between objectsNetworks exist in the real world. (Data)Soci

6、al networks, logistic networks, biology networks, transaction networks, etc.A network can be represented by a graph.A dataset that is not a network can also be represented by a graph.图卷积网络应用于自然语言处理Many papers on BERT + GNN.BERT is for retrieval.It creates an initial graph of relevant entities and th

7、e initial evidence.GNN is for reasoning.It collects evidence (i.e., old messages on the entities) and arrive at new conclusions (i.e., new messages on the entities), by passing the messages around and aggregating them.Cognitive Graph for Multi-Hop Reading Comprehension at Scale. Ding et al., ACL 201

8、9.Dynamically Fused Graph Network for Multi-hop Reasoning. Xiao et al., ACL 2019.图卷积网络应用于计算机视觉A popular trend in CV is to construct a graph during the learning process.To process multiple objects or parts in a scene, and to infer their relationships.Example: Scene graphs.Scene Graph Generation by It

9、erative Message Passing. Xu et al., CVPR 2017.Image Generation from Scene Graphs. Johnson et al., CVPR 2018.图卷积网络应用于符号推理We can view the process of symbolic reasoning as a directed acyclic graph.Many recent efforts use GNNs to perform symbolic reasoning.Learning by Abstraction: The Neural State Machi

10、ne. Hudson & Manning, 2019.Can Graph Neural Networks Help Logic Reasoning? Zhang et al., 2019.Symbolic Graph Reasoning Meets Convolutions. Liang et al., NeurIPS 2018.Structural equation modeling, a form of causal modeling, tries to describe the relationships between the variables as a directed acycl

11、ic graph (DAG).GNN can be used to represent a nonlinear structural equation and help find the DAG, after treating the adjacency matrix as parameters.图卷积网络应用于结构方程建模DAG-GNN: DAG Structure Learning with Graph Neural Networks. Yu et al., ICML 2019.（大多数）图卷积网络方法的PipelineCo-occurrence (neighborhood)网络嵌 : 拓

12、扑向量化High-order proximities网络嵌 : 拓扑向量化Communities网络嵌 : 拓扑向量化Heterogeneous networks网络嵌 : 拓扑向量化（大多数）网络嵌入方法的PipelineLearning for Networks v.s. Learning via GraphsLearning for networksLearning Via GraphsNetwork EmbeddingGCN网络嵌入方法解决的核心问题Node NeighborhoodCommunityPair-wise ProximityHyper EdgesGlobal Struct

13、ureReducing representation dimensionality while preserving necessary topologicalstructures and properties.Nodes & LinksNon-transitivityAsymmetric TransitivityDynamicUncertaintyHeterogeneityInterpretabilityTopology-driven图卷积神经网络方法解决的核心问题Fusing topologyand featuresin the wayof smoothing features with

14、the assistance of topology.Feature-driven如果问题是拓扑驱动的？Since GCN is filtering features, it is inevitably feature-drivenStructure only provides auxiliary information (e.g. for filtering/smoothing)When feature plays the key role, GNN performs good How about the contrary?Synthesis data: stochastic block m

15、odel + random featuresMethodResultsRandom10.0GCN18.31.1DeepWalk99.00.1网络嵌入 v . 图神经网络There is no better one, but there is more proper one.反思：图神经网络是否真的是深度学习方法？Recall GNN formulation:= &!(!#$, = +*,$/.0/+*,$/.How about removing the non-linear component:!#$= !(Stacking multiple layers and add softmax classification:21= 3456789!:= 3456789 !($ (:,$= 3456789:!(High-order proximityWu, Felix, et al. Simplifying graph convolutional networks. ICML, 2019.Thissimplified GNN (SGC) showsremarkable results:Node classificationText Classification反思：图神经网络是否真的是深度学习方法？Wu