We compare the number density of compact (small size) massive galaxies at low and high redshift using our Padova Millennium Galaxy and Group Catalogue (PM2GC) at z = 0.03-0.11 and the CANDELS results from Barro et al. at z = 1-2. The number density of local compact galaxies with luminosity weighted (LW) ages compatible with being already passive at high redshift is compared with the density of compact passive galaxies observed at high-z. Our results place an upper limit of a factor ~2 on the evolution of the number density and are inconsistent with a significant size evolution for most of the compact galaxies observed at high-z. Instead, the evolution may be significant (up to a factor five) for the most extreme, ultracompact galaxies. Considering all compact galaxies, regardless of LW age and star formation activity, a minority of local compact galaxies (<=1/3) might have formed at z < 1. Finally, we show that the secular decrease of the galaxy stellar mass due to simple stellar evolution may in some cases be a non-negligible factor in the context of the evolution of the mass-size relation, and we caution that passive evolution in mass should be taken into account when comparing samples at different redshifts.