PropInit: Scalable inductive initialization for heterogeneous graph neural networks

Graph Neural Networks (GNNs) require that all nodes have initial representations which are usually derived from the node features. When the node features are absent, GNNs can learn node embeddings with an embedding layer or use pretrained network embeddings for the initial node representations. However, these approaches are limited because i) they cannot be easily extended to initialize new nodes that are added to the graph for inference after training and ii) they are memory intensive and store a fixed representation for every node in the graph.

In this work, we present PropInit a scalable node representation initialization method for training GNNs and other Graph Machine Learning (ML) models on heterogeneous graphs where some or all node types have no natural features. Unlike existing methods that learn a fixed embedding vector for each node, PropInit learns an inductive function that leverages the metagraph to initialize node representations.

As a result, PropInit is fully inductive and can be applied, without retraining, to new nodes without features that are added to the graph. PropInit also scales to large graphs as it requires only a small fraction of the memory requirements of existing methods. On public benchmark heterogeneous graph datasets, using various GNN models, PropInit achieves comparable or better performance to other competing approaches while needing only 0.01% to 2% of their memory consumption for representing node embeddings. We also demonstrate PropInit’s effectiveness on an industry heterogeneous graph dataset for fraud detection and achieve better classification accuracy than learning full embeddings while reducing the embedding memory footprint during training and inference by 99.99%