In Section 3.4.1 we have this definition -These three matrices are used to project the embedded input tokens, x(i), into query, key, and value vectors, respectively, as illustrated in figure 3.14

These weight matrices are then initialized later to random tensors like so:

torch.manual_seed(123)W_query = torch.nn.Parameter(torch.rand(d_in, d_out), requires_grad=False)W_key   = torch.nn.Parameter(torch.rand(d_in, d_out), requires_grad=False)W_value = torch.nn.Parameter(torch.rand(d_in, d_out), requires_grad=False)

• Where then or how is the input being projected then to these respective weight matrices?
• Is the intuition behind the split to also reduce dimensionality?