Let's start with our cost function:

$$\mathcal{L}(A_{ij}) = \sum_{i=1}^{N_\mathrm{out}} (z_i - y_i^k)^2 = \sum_{i=1}^{N_\mathrm{out}}

$$\mathcal{L}(A_{ij}) = \sum_{i=1}^{N_\mathrm{out}} (z_i - y_i^k)^2 = \sum_{i=1}^{N_\mathrm{out}}

where we'll refer to the product ${\boldsymbol \alpha} \equiv {\bf

$$\frac{\partial \mathcal{L}}{\partial A_{pq}} =

with

and for $g(\xi)$, we will assume the sigmoid function,so

$$\frac{\partial g}{\partial \xi}

$$\frac{\partial g}{\partial \xi}

(z_i - y^k_i) z_i (1 - z_i) \delta_{ip} x^k_q\\

&= 2 (z_p - y^k_p) z_p (1- z_p) x^k_q

\end{align*}

where we used the fact that the $\delta_{ip}$ means that only a single term contributes to the sum.

Now ${\bf z}$ and ${\bf y}^k$ are all vectors of size $N_\mathrm{out} \times 1$ and ${\bf x}^k$ is a vector of size $N_\mathrm{in} \times 1$, so we can write this expression for the matrix as a whole as:

where the operator $\circ$ represents_element-by-element_ multiplication (the[Hadamard product](https://en.wikipedia.org/wiki/Hadamard_product_(matrices))).

The overall minimization appears as:

</div>