Hello, I am a beginner in GAT , and I've been studying your GATv2 code lately. I have a question while going through the code in

labml_nn/graphs/gatv2/init.py

When calculatingg_sum

g_sum = g_l_repeat + g_r_repeat_interleave

You mentioned in the comments: Now we add the two tensors to get

$$\lbrace\overrightarrow{g_{l1}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l1}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{l1}} + \overrightarrow{g_{rN}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{l2}} + \overrightarrow{g_{rN}}, \dots\rbrace$$

But in the previous code,g_l_repeat gets

$$\lbrace\overrightarrow{g_{l1}}, \overrightarrow{g_{l2}}, \dots, \overrightarrow{g_{lN}}, \overrightarrow{g_{l1}}, \overrightarrow{g_{l2}}, \dots, \overrightarrow{g_{lN}}, \dots\rbrace$$

andg_r_repeat_interleave gets

$$\lbrace\overrightarrow{g_{r1}}, \overrightarrow{g_{r1}}, \dots, \overrightarrow{g_{r1}}, \overrightarrow{g_{r2}}, \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{r2}}, \dots\rbrace$$

So I think the result of adding the two tensors should be

$$\lbrace\overrightarrow{g_{l1}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{lN}} + \overrightarrow{g_{r1}}, \overrightarrow{g_{l1}} + \overrightarrow{g_{r2}}, \overrightarrow{g_{l2}} + \overrightarrow{g_{r2}}, \dots, \overrightarrow{g_{lN}} + \overrightarrow{g_{r2}}, \dots\rbrace$$

I'm not sure whether I may have overlooked some crucial information or if there's a mismatch between your comments and the code. I would greatly appreciate it if you could help clarify my confusion. Thank you.