We analyse age-velocity dispersion relations (AVRs) from kinematics of individual stars in eight Local Group galaxies ranging in mass from Carina (M* ˜ 106 M⊙) to M31 (M* ˜ 1011 M⊙). Observationally the σ versus stellar age trends can be interpreted as dynamical heating of the stars by giant molecular clouds, bars/spiral arms or merging subhaloes; alternatively the stars could have simply been born out of a more turbulent interstellar medium (ISM) at high redshift and retain that larger velocity dispersion till present day - consistent with recent integral field unit kinematic studies. To ascertain the dominant mechanism and better understand the impact of instabilities and feedback, we develop models based on observed star formation histories (SFHs) of these Local Group galaxies in order to create an evolutionary formalism that describes the ISM velocity dispersion due to a galaxy's evolving gas fraction. These empirical models relax the common assumption that the stars are born from gas that has constant velocity dispersion at all redshifts. Using only the observed SFHs as input, the ISM velocity dispersion and a mid-plane scattering model fits the observed AVRs of low-mass galaxies without fine tuning. Higher mass galaxies above Mvir ≳ 1011 M⊙ need a larger contribution from latent dynamical heating processes (for example minor mergers), in excess of the ISM model. Using the SFHs, we also find that supernovae feedback does not appear to be a dominant driver of the gas velocity dispersion compared to gravitational instabilities - at least for dispersions σ ≳ 25 km s-1. Together our results point to stars being born with a velocity dispersion close to that of the gas at the time of their formation, with latent dynamical heating operating with a galaxy mass-dependent efficiency. These semi-empirical relations may help constrain the efficiency of feedback and its impact on the physics of disc settling in galaxy formation simulations.