An isolated, initially cold and ellipsoidal cloud of self-gravitating particles represents a relatively simple system in which to study the effects of deviations from spherical symmetry in the mechanism of violent relaxation. Initial deviations from spherical symmetry are shown to play a dynamical role that is equivalent to that of density fluctuations in the case of an initially spherical cloud. Indeed, these deviations control the amount of particle-energy change and thus determine the properties of the final energy distribution, particularly the appearance of two species of particles: bound and free. Ejection of mass and energy from the system, together with the formation of a density profile decaying as ρ(r) ∼ r−4 and a Keplerian radial velocity dispersion profile, are prominent features similar to those observed after the violent relaxation of spherical clouds. In addition, we find that ejected particles are characterized by highly non-spherical shapes, the features of which can be traced in the initial deviations from spherical symmetry that are amplified during the dynamical evolution: particles can indeed form anisotropic configurations, like bars and/or discs, even though the initial cloud was very close to spherical.