It has recently been shown that in a broad class of disordered systems oscillatory shear training can embed memories of specific shear protocols in relevant physical parameters such as the yield strain. These shear protocols can be used to change the physical properties of the system and memories of the protocol can later be "read" out. Here we investigate shear training memories in colloidal gels, which include an attractive interaction and network structure, and discover that such systems can support memories both along and orthogonal to the training flow direction. We use oscillatory shear protocols to set and read out the yield strain memories and confocal microscopy to analyze the rearranging gel structure throughout the shear training. We find that the gel bonds remain largely isotropic in the shear-vorticity plane throughout the training process suggesting that structures formed to support shear along the training shear plane are also able to support shear along the orthogonal plane. Orthogonal memory extends the usefulness of shear memories to more applications and should apply to many other disordered systems as well. This journal is © The Royal Society of Chemistry.