We present the result of an investigation into the optical trapping of micropaticles using laser beams with a spatially inhomogeneous polarization (cylindrical vector beams). We perform three-dimensional tracking of the Brownian fluctuations in position of a trapped particle and extract the trap spring constants. We characterize the trap geometry by the aspect ratio of spring constants in the directions transverse and parallel to the beam propagation direction and evaluate this figure of merit as a function of polarization angle. We show that the additional degree of freedom present in cylindrical vector beams (CVBs) allows us to control the optical trap strength and geometry by adjusting the polarization of the trapping beam only. Experimental results are compared with a theoretical model of optical trapping using CVBs derived from electromagnetic scattering theory in the T-matrix framework.