Distance encoding module

Following refs. ^54,57, we also leverage the distances between pairs of nodes to further encode the spatial features of the molecular line graphs. More specifically, given nodes v^i and v^j in a molecular line graph, we encode their distance to an attention scalar ai,jd in a distance attention matrix Ad∈RNv^×Nv^ as follows:

where SPD(⋅) stands for the shortest path distance functoin, d_i,j stands for the derived distance between v^i and v^j, W1d∈RDd×1 and W2d∈R1×Dd stand for the trainable projection matrices, and D_d stands for the dimension of the distance embedding.

Then, to introduce the encoded structural information into the model, we rewrite the formula of the attention matrix Al,k∈RNv^×Nv^ in the Eq. (2) as follows:

where A^p and A^d are the path encoding matrix and the distance encoding matrix, respectively.

Here, we discuss the main advantages of our proposed model compared with the previously defined graph transformers:

First, by representing molecular graphs as line graphs, LiGhT emphasizes the importance of chemical bonds in molecules. Chemical bonds are the lasting attractions between atoms, which can be categorized into various types according to the ways they hold atoms together, resulting in different properties of the formed molecules. However, the previously defined transformer architectures either omit the edge features or only introduce chemical bonds as the bias in the self-attention module, ignoring the rich information from chemical bonds^53–57. In our case, LiGhT fills this gap and fully exploits the intrinsic features of chemical bonds.

Second, although strategies like path encoding have already been proposed in previous graph transformer architectures^53,57 when encoding the paths, they only consider the edge features and ignore the node features in the paths. On the other hand, our path encoding strategy incorporates the features of the complete paths between pairs of nodes, thus encoding the structural information more precisely compared to the previous methods.

In summary, LiGhT provides a reliable backbone network for accurately modeling the structural and semantic information of molecular line graphs.