MELL: Effective Embedding Method for Multiplex Networks - PowerPoint Presentation

MELL: Effective Embedding Method for Multiplex Networks International workshop on Mining Attributed NetworksLyon, 23 April 2018 Ryuta Matsuno1,2, Tsuyoshi Murata11Tokyo Institute of Technology, Tokyo, Japan2e-mail: ryutamatsuno@net.c.titech.ac.jp

In This Work Study about “embedding method for multiplex networks”.Propose new embedding method, MELL, which uses layer vectors to capture the layer structures.Link prediction experiments show MELL outperforms existing baseline methods. 2

Contents IntroductionRelated Works Our Method: MELLExperimentsConclusion3

1. Introduction 4

Network Embedding Network embedding is a method for converting nodes in a network into low dimensional vectors, preserving the network structure.Many methods have been proposed, e.g., DeepWalk [Perozzi et al., 2014], LINE [Tang et al., 2015] and node2vec [Grover et al, 2016]. 5 B A C D B A C D Network Embedding (representation Learning) Machine Learning Tasks (classification, link prediction)

Background The real world networks have several types of connections. 6 B A C D B A C D B A C D Multiplex Network A B C D A B C D A B C D Layer 1 Layer 2 Layer 3

Background Most of embedding methods are designed for single-layer networks.Thus this research aims at developing an embedding method for multiplex networks, and we test our method by link prediction tasks. 7 Layer 1 Layer 2 Layer 3 Our Research Objective Develop an Embedding method & Test by link prediction tasks

Problem Formulation Link prediction in multiplex network problemGiven a multiplex network, it aims at calculating edge probabilities that there are potential edges between unconnected node pairs in the network. 8 Input : Output : Multiplex Network Edge probabilities Layer 1, A-D 0.12 Layer 1, B-C 0.27 Layer 1, C-D 0.02 Layer 2, B-D 0.33 Layer 3, B-C 0.86 …

Challenge How to use other layer structuresSimilar layers should be considered at the same time.Even different layers might have useful information. 9 Social Type A B C D A B C D A B C D Transportation Type

2. Related Works 10

2. Related Works APP [Zhou 2017]It is an embedding method for single layer networks, and it predicts links based on embedding results. For multiplex network, it embeds each layer independently.MTNE [Xu 2017]This is an embedding method designed for multiplex networks. It embeds each layer, and improve them by the consensus embedding among all layers. MULTITENSOR [De Bacco 2017]This method is based on the Poisson tensor factorization. This decomposes the adjacency tensor, and regenerates it for link prediction.11

3. Our Method: MELL 12

Our Method : MELL MELL = Multiplex network Embedding method via Learning Layer vector 13 Ideas MELL enforces the vectors for the same node in the layers to be close to each other (enforcing). MELL uses layer vector.

Idea 1 : Enforcing MELL embeds each layer. Then it enforces the vectors for the same node to be close to each other. 14 A B D C Embed each layer Enforce same nodes to be close A C B D C B A D B D B A D A C C B-C Edge probability Layer 1 : 0.92 Layer 2 : 0.95 Layer 3 : 0.93 Calculate Edge probability Embedding results Good : It predicts edges for similar layers. Bad : It predicts unreasonable edges when layers are different.

Idea 2 : Layer vector MELL uses layer vectors which capture layer similarities based on the layer structures.Layer vectors are used to calculate the edge probabilities.This makes it possible to predict links considering the layer similarities or differences. 15Layer Vector R A C B D B-C edge probability Layer 1 : 0.24 Layer 2 : 0.98 Layer 3 : 0.86 Node vector Embed With layer vector Calculate edge probabilities With layer vectors Embedding results

Edge Probability The edge probability from i-th node to j- th node is defined as follows:16

Loss Function The final loss function is defined as follows: 17 where denote the head and tail embedding tensor respectively, which contain all head and tail embedding vectors. denotes the layer vectors. denotes the all existing edges in the given multiplex network. is the set of negative samples.

18

4. Experiments 19

4. Experiments We test MELL and baselines for link prediction tasks using 5 types of data sets.αt% of the edges are used for training, and the others are used for testing.AUC (AUROC) is used for the evaluation. 20

Results 21 CS-Aarhus Pierre Auger Collaboration EU Air Transportation Xenopus C.Elegans Connectome

5. Conclusion 22

Conclusion We proposed MELL, which predicts links in multiplex networks better than all of the baseline methods.We propose layer vector, which captures the layer connectivity.Future workScalability Apply MELL to temporal networks.23

References [Perozzi et al. 2014] Perozzi, Bryan ; Al-Rfou, Rami ; Skiena, Steven: DeepWalk: Online Learning of Social Representations. In: 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014 (KDD ’14)[Tang et al. 2015] Tang, Jian ; Qu, Meng ; Wang, Mingzhe ; Zhang, Ming ; Yan, Jun ; Mei, Qiaozhu: LINE: Large-scale Information Network Embedding. In: 24th International Conference on World Wide Web, 2015 (WWW ’15)[Grover et al. 2016] Grover, Aditya ; Leskovec, Jure: node2Vec: Scalable Feature Learning for Networks. In: 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016 (KDD ’16)[De Bacco et al. 2017] De Bacco , Caterina ; Power, Eleanor A. ; Larremore , Daniel B. ; Moore, Cristopher: Community detection, link prediction, and layer interdependence in multilayer networks. In: Physical Review E 95, 2017 [Xu et al. 2017] Xu, Linchuan ; Wei, Xiaokai ; Cao, Jiannong ; Yu, Philip S. : Multi-task Network Embedding. In: 2017 IEEE International Conference on Data Science and Advanced Analytics, 2017 (DSAA ’17)[Zhou et al. 2017] Zhou, Chang ; Liu, Yuqiong ; Liu, Xiaofei ; Liu, Zhongyi ; Gao, Jun: Scalable Graph Embedding for Asymmetric Proximity. In: 31st AAAI Conference on Artificial Intelligence, 2017 (AAAI ’17). 24

25

Experimental Setting All existing edges are split into five parts.Depends on αt, one to four parts are used for training. Other parts are used for testing with the same number of unconnected node pairs, sampled randomly.Picking pairs are changed five times. 26 All existing edges Split Training α t = 40% Testing All unconnected node pairs Sample

Inner product method MELL is based on “inner product method” for single layer networks.The edge probability is defined as follows:Loss function is defined as follows: 27

Idea 1 : Enforcing Enforcing is implemented by adding “enforcing term” to the loss function. 28 where denotes the embedding vectors. is embedding vectors for l - th layer. L denotes the no. of layers. N denotes the no. of nodes.

Idea 2 : Layer vector Layer vectors are added to the head node in edge probability. 29 where denotes the embedding vector for i -th node in l - th layer. denotes the layer vector for l - th layer.

Idea 3 : Two vectors for each node For undirected network, we use two vectors for each node; one is as a head and the other is as a tail.The edge probability from i -th node to j-th node is defined as follows:30 The subscripts H and T stand for Head and Tail, respectively.

Parameter Sensitivity 31

Layer vector analysis The similarity between a pair of layer structures is defined as follows:The similarity between a pair of layer vectors is defined as follows: 32

Layer vector analysis 33 CS-AarhusPierre Auger Collaboration EU Air Transportation Xenopus C.Elegans Connectome

CS-Aarhus Number of layers : 5Number of nodes : 61Number of total edges : 620Type : socialDirected : NoNode : person Explanations : This undirected multiplex social network consists of five kinds of online and offline relationships (Facebook, Leisure, Work, Co-authorship, Lunch) between the employees of Computer Science department at Aarhus.34

Pierre Auger Collaboration Number of layers : 16Number of nodes : 514Number of total edges : 7153Type : co-authorshipDirected : NoNode : person Explanations : This undirected co-authorship multiplex network consists of 16 types of different working tasks within the Pierre Auger Collaboration. The layers represents Neutrinos, Detector, Enhancements and other tasks. This network is originally weighted, but the weights are ignored for the experiments in the thesis.35

EU Air Transportation Number of layers : 37Number of nodes : 450Number of total edges : 3588Type : transportationDirected : NoNode : airport Explanations : This undirected transportation multiplex network is composed by 37 different layers each one corresponding to a different airline operating in Europe. The nodes are airports, and the edges are routes, respectively.36

C.Elegans Connectome Number of layers : 3Number of nodes : 279Number of total edges : 5863Type : neuronalDirected : YesNode : neuronal cell Explanations : This directed neuronal multiplex network, Caenorhabditis Elegans connectome, consists of three different synaptic junctions: electric, chemical monadic, and polyadic.37

Xenopus Number of layers : 5Number of nodes : 461Number of total edges : 620Type : geneticDirected : Yes Node : genes/proteinsExplanations : This directed genetic multiplex network consists of five types of different interactions for organisms: association, direct interaction, physical association, colocalization, and suppressive genetic interaction defined by inequality. The Biological General Repository for Interaction Data sets 3.2.108 (BioGRID, thebiogrid.org, updated 1 Jan 2014) is used, and Xenopus laevis is concerned.38

APP 39

MTNE 40

MULTITENSOR 41

AUC (AUROC) AUC stands for Area Under the Curve, and AUROC stands for Area Under ROC curveAUC shows the probability that a randomly chosen positive sample will produce a score higher than the score of a randomly chosen negative sample. 42 FP rate TP rate This area shows AUC ( AUROC).

Time Complexity 43 Environment : Ubuntu 16.04, Core i9-7900X, 64GB memory, NVIDIA GeForce GTX1080 Ti

Effectiveness of Layer Vector 44 44 CS-Aarhus Pierre Auger Collaboration EU Air Transportation Xenopus C.Elegans Connectome

Layer Statistics 45

ROC Curve 46 CS-Aarhus Pierre Auger Collaboration EU Air Transportation Xenopus C.Elegans Connectome

Prediction Example (EU Air Transportation) 47 MELL MULTITENSOR