Abstract
Self-supervised learning in graphs has emerged as a promising avenue for harnessing unlabeled graph data, leveraging pretext tasks to generate informative node representations. However, the reliance on a single pretext task often constrains generalization across various downstream tasks and datasets. Recent advancements in multi-task learning on graphs aim to tackle this limitation by integrating multiple pretext tasks, framing the problem as a multi-objective optimization to train a shared set of parameters. However, these approaches frequently encounter task interference, where competing tasks degrade overall performance by conflicting with each other due to the limited expressivity of the model. In this work, we introduce MEGA, a novel multi-encoder graph neural network architecture designed to alleviate task interference by providing distinct parameter spaces for the decoupled training of each task. This architecture allows for independent learning from multiple pretext tasks, followed by a simple self-supervised dimensionality reduction technique to combine the insights gleaned. Through extensive experiments, we demonstrate the superiority of our approach, showcasing an average performance improvement of $$3.8backslash%$$3.8%across three commonly used downstream tasks (i.e., link prediction, node classification, and partition prediction) and nine benchmark datasets.
