The relation between galaxies and dark matter haloes is of vital importance for evaluating theoretical predictions of structure formation and galaxy formation physics. We use hydrodynamic cosmological simulations of the Local Group to show that the widely used method of abundance matching based on dark matter only simulations fails at the low-mass end because two of its underlying assumptions are broken: only a small fraction of low-mass (<109.5 M⊙) haloes host a visible galaxy, and baryon effects lower their growth rate. In this regime, reliance on dark matter only simulations for abundance matching is inappropriate and can lead to incorrect results. We find that the reported discrepancy between observational estimates of the halo masses of dwarf galaxies and the values predicted by abundance matching does not point to a failure of Λ cold dark matter, but simply to a failure to account for baryonic effects. Our results also imply that the Local Group contains only a few hundred observable galaxies in contrast with the thousands of faint dwarfs that abundance matching would suggest. We show how relations derived from abundance matching can be corrected, so that they can be used self-consistently to calibrate models of galaxy formation.