Potential Energy to Improve Link Prediction with Relational Graph Neural Networks

This repository contains the code used for the experiments in the paper.

Potential Energy to Improve Link Prediction with Relational Graph Neural Networks.
Simone Colombo, Dimitris Alivanistos and Michael Cochez

Abstract: Potential Energy (PE) between 2 bodies with mass, refers to the relative gravitational pull between them. Analogously, in the context of a graph, nodes can thought of as objects where a) the product of the degrees of nodes acts as a proxy for mass, b) the clustering coefficients of common neighbours as a proxy for gravitational acceleration, and c) the inverse of the shortest distance between nodes as a proxy for distance in space, which allows for PE calculation as introduced in prior work. In this work, we are investigating the effects of incorporating PE in Link Prediction (LP) with Relational Graph Convolutional Networks (R-GCN). Specifically, we explore the benefits of including PE calculation as an informative prior to the LP task and in a follow-up experiment as a learnable feature to predict. We performed several experiments and show that considering PE in the LP process has certain advantages and find that the information PE provides was not captured by the embeddings produced by the R-GCN.

Model descriptions

Vanilla R-GCN: The R-GCN for LP with an additional Dense Layer to reduce the dimension of the textual embeddings

eR-GCN: The energized R-GCN - utilizing the additional signal of PE for prediction, note that PE comes directly from the input graph topology, also note the trainable parameter 𝛼 that represents signal weight.

PE Estimator: An MLP-Regressor trained to predict the PE between nodes.

eR-GCN-MLP: The combination of eR-GCN and PE estimator that optimizes 𝛼 based on both the R-GCN and the PE Estimator. The light blue box indicates frozen parameters, which allows us to isolate the effects of specific parts in the architecture.

Installation

Conda Installation

1. The general installation instruction for Conda can be found here.
   1. For Windows users, the specific instructions can be found here
   2. For Mac OS X users, the specific instructions can be found here
      • If you are on Mac and using the zsh with iTerm2,then, follow this guide if the jupyter conda command is not found.
      • Another option for MAC OS X user would be running the followinf script:

   wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh\
    -O ~/miniconda.sh
    bash ~/miniconda.sh -b -p $HOME/miniconda
   

   3.For Linux users, the specific instructions can be found here, or, the following script can be run:

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\
  -O ~/miniconda.sh
  bash ~/miniconda.sh -b -p $HOME/miniconda

4. For Linux and Mac users only, to run 'conda' from the terminal, run 'export PATH="$HOME/miniconda/bin:$PATH"'. If you want do not want to run it each time you start the system, add it to ~/.bashrc for Linux users or to ~/.zshrc for Mac OS X users. Then, run source ~/.bashrc (or ~/.zshrc) to update it.
5. After the installation of conda, you should verify that it has been installed correctly by running the command 'conda list' in your terminal. If it runs, it should return a list of all packages installed:

packages in environment at /opt/anaconda3/envs/pyg:

Name                    Version                   Build  Channel
ase                       3.21.1                   pypi_0    pypi
blas                      1.0                         mkl  
ca-certificates           2021.4.13            hecd8cb5_1  
cached-property           1.5.2                    pypi_0    pypi
cctools                   927.0.2              h5ba7a2e_4  
certifi                   2020.12.5        py36hecd8cb5_0  
chardet                   4.0.0                    pypi_0    pypi')
...                         ...                       ...

6.Then, update conda and initialize the conda environment by running

conda init
conda update conda
conda create -n conda3.6 python=3.6
source activate conda3.6

PyTorch Geometric Installation

2. The general installation instruction for pytorch_geometric can be found here.
   1. As first step, at least PyTorch 1.4.0 has to be installed. If it is not installed, there are different option for different users:
      • If you are a Windows user, this guide should lead you to a successful pytorch installation with conda.
      • If you are a Linux or Mac OS X user you can run 'conda install pytorch torchvision -c pytorch' in the terminal to install it.
   2. As second step, you can follow the general instruction in the link at 2).
3. For Mac OS X users only, it is possible to use the script pyg_setup.sh by running 'sh ./pyg_setup.sh' in the terminal and this will create the conda environment, activate it, install pytorch and install all the necessary dependencies for pytorch_geometric.

Run the code

Run the RGCN with MLP

In the folder src, run the following python files in the sequence below as follows:

• python3 train_standard_RGCN.py (in order to train the vanilla RGCN)
• python3 train_PE_estimator.py (in order to train the PE_estimator only by freezing the vanilla RGCN)
• python3 train_R_GCN_MLP.py (in order to train the RGCN with the MLP)

Run the eRGCN

• python3 train_standard_RGCN.py (in order to train the vanilla RGCN)
• python3 train_eRGCN.py (in order to train the eRGCN)