GOAT: A global transformer on large-scale graphs
A new scalable global graph transformer model that effectively handles both homophilious and heterophilious graphs.
Graph transformers have been competitive on graph classification tasks, but they fail to outperform Graph Neural Networks (GNNs) on node classification, which is a common task performed on large-scale graphs for industrial applications. Meanwhile, existing GNN architectures are limited in their ability to perform equally well on both homophilious and heterophilious graphs as their inductive biases are generally tailored to only one setting. To address these issues, we propose GOAT, a scalable global graph transformer. In GOAT, each node conceptually attends to all the nodes in the graph and homophily/heterophily relationships can be learned adaptively from the data. We provide theoretical justification for our approximate global self-attention scheme and show it to be scalable to large-scale graphs. We demonstrate the competitiveness of GOAT on both heterophilious and homophilious graphs with millions of nodes. We open source our implementation at https://github.com/devnkong/GOAT.
Latest publications
LLM-SRBench: A new benchmark for scientific equation discovery with Large Language Models
A comprehensive benchmark designed to evaluate LLM-based scientific equation discovery methods.
ICMLZero-shot meta-learning for tabular prediction tasks with adversarially pre-trained transformer
Introducing APT, an Adversarially Pre-trained Transformer achieving SOTA on small tabular tasks.
ICMLDynamic customer embeddings for financial services applications
Learning dynamic customer representations from digital activity and financial context.
ICML