You are in the right here. Horizontal vectors are common for (some) deep learning docs, but column factors are the literature standard elsewhere.