GOAT: A global transformer on large-scale graphs
A new scalable global graph transformer model that effectively handles both homophilious and heterophilious graphs.
Graph transformers have been competitive on graph classification tasks, but they fail to outperform Graph Neural Networks (GNNs) on node classification, which is a common task performed on large-scale graphs for industrial applications. Meanwhile, existing GNN architectures are limited in their ability to perform equally well on both homophilious and heterophilious graphs as their inductive biases are generally tailored to only one setting. To address these issues, we propose GOAT, a scalable global graph transformer. In GOAT, each node conceptually attends to all the nodes in the graph and homophily/heterophily relationships can be learned adaptively from the data. We provide theoretical justification for our approximate global self-attention scheme and show it to be scalable to large-scale graphs. We demonstrate the competitiveness of GOAT on both heterophilious and homophilious graphs with millions of nodes. We open source our implementation at https://github.com/devnkong/GOAT.
Latest publications
Ask, don’t judge: Binary questions for interpretable LLM evaluation and self-improvement
A framework that decomposes evaluation criteria into atomic binary questions and aggregates the resulting verdicts.
ICMLStress tests REVEAL fragile temporal and visual grounding in video-language models
A diagnostic benchmark that probes fundamental weaknesses of contemporary VidLMs through five controlled stress tests.
ICMLSEAD: Competence-aware on-policy distillation via entropy-guided supervision
A framework which uses entropy as a unified probe of this competence-dependent degradation at three scales.
ICML