We present high-quality, wide spectral coverage long-slit optical spectra for 12 powerful radio sources at low and intermediate redshifts (z < 0.7) that show evidence for a substantial ultraviolet (UV) excess. These data were taken using the William Herschel Telescope and the ESO Very Large Telescope with the aim of determining the detailed properties of the young stellar populations (YSPs) in the host galaxies as part of a larger project to investigate evolutionary scenarios for the active galactic nucleus (AGN) host galaxies. The results of our spectral synthesis model fits to the spectra highlight the importance of taking into account AGN-related components (emission lines, nebular continuum, scattered light) and reddening of the stellar populations in studies of this type. It is also clear that careful examination of the fits to the spectra, as well consideration of auxiliary polarimetric and imaging data, are required to avoid degeneracies in the model solutions. In three out of the 12 sources in our sample we find evidence for broad permitted line components, and a combination of AGN-related continuum components and an old (12.5 Gyr) stellar population provides an adequate fit to the data. However, for the remaining nine sources we find strong evidence for YSPs. In contrast to some recent studies that suggest relatively old post-starburst ages for the YSPs in radio galaxies (0.3-2.5 Gyr), we deduce a wide range of ages for the YSPs in our sample objects (0.02-1.5 Gyr), with ~50 per cent of the sample showing evidence for young YSP ages (<~0.1 Gyr) in their nuclear regions. The nuclear YSPs are often significantly reddened [0.2 < E(B - V) < 1.4] and make up a substantial fraction (~1-35 per cent) of the total stellar mass in the regions sampled by the spectroscopic slits. Moreover, in all the cases in which we have sufficient spatial resolution we find that the UV excess is extended across the full measurable extent of the galaxy (typically 5-30 kpc), suggesting galaxy-wide starbursts. The implications for photometric and spectroscopic studies of active galaxies are discussed.