We present spectra and near-infrared images of a sample of faint radio sources initially selected as promising high-redshift radio galaxy (HzRG) candidates. We have determined redshifts for a total of 13 radio galaxies with redshifts ranging from 0.52 ≤ z ≤ 5.72. Our sample probes radio luminosities that are almost an order of magnitude fainter than previous large samples at the highest redshifts. We use near-infrared photometry for a subsample of these galaxies to calculate stellar masses using simple stellar population models, and find stellar masses to be in the range 10^{10.8} {-}10^{11.7} M_☉. We then compare our faint radio galaxies with brighter radio galaxies at z ≥ 2 from the literature. We find that fainter radio galaxies have lower Ly α luminosities and narrower line widths compared to the bright ones, implying photoionization by a weaker active galactic nucleus (AGN). We also rule out the presence of strong shocks in faint HzRGs. The stellar masses determined for faint HzRGs are lower than those observed for brighter ones. We find that faint HzRG population in the redshift range 2-4 forms a bridge between star-forming and narrow-line AGNs, whereas the ones at z > 4 are likely to be dominated by star formation, and may be building up their stellar mass through cold accretion of gas. Finally, we show that the overall redshift evolution of radio sizes at z > 2 is fully compatible with increased inverse Compton scattering losses at high redshifts.