Tracing the emergence of the massive quiescent galaxy (QG) population requires the build-up of reliable quenched samples by distinguishing these systems from red, dusty star-forming sources. We present Hubble Space Telescope WFC3/G141 grism spectra of ten quiescent galaxy candidates selected at 2.5 < z < 3.5 in the COSMOS field. Spectroscopic confirmation for the whole sample is obtained within one to three orbits through the detection of strong spectral breaks and Balmer absorption lines. When their spectra are combined with optical to near-infrared photometry, star-forming solutions are formally rejected for the entire sample. Broad spectral indices are consistent with the presence of young A-type stars, which indicates that the last major episode of star formation has taken place no earlier than ∼300-800 Myr prior to observation. This confirms clues from their post-starburst UVJ colors. Marginalising over three different slopes of the dust attenuation curve, we obtain young mass-weighted ages and an average peak star formation rate (SFR) of ∼103 M⊙ yr−1 at zformation ∼ 3.5. Although mid- and far-IR data are too shallow to determine the obscured SFR on a galaxy-by-galaxy basis, the mean stacked emission from 3 GHz data constrains the level of residual-obscured SFR to be globally below 50 M⊙ yr−1, three times below the scatter of the coeval main sequence. Alternatively, the very same radio detection suggests a widespread radio-mode feedback by active galactic nuclei (AGN) four times stronger than in z ∼ 1.8 massive QGs. This is accompanied by a 30% fraction of X-ray luminous AGN with a black hole accretion rate per unit SFR enhanced by a factor of ∼30 with respect to similarly massive QGs at lower redshift. The average compact, high Sérsic index morphologies of the galaxies in this sample, coupled with their young mass-weighted ages, suggest that the mechanisms responsible for the development of a spheroidal component might be concomitant with (or preceding) those causing their quenching.