In this paper, we present a new derivation of the shape and evolution of the integrated galaxy-wide initial mass function (IGIMF), incorporating explicitly the effects of cosmic rays (CRs) as regulators of the chemical and thermal state of the gas in the dense cores of molecular clouds. We predict the shape of the IGIMF as a function of star formation rate and CR density and show that it can be significantly different with respect to local estimates. In particular, we focus on the physical conditions corresponding to IGIMF shapes that are simultaneously shallower at high-mass end and steeper at the low-mass end than those of a Kroupa IMF. These solutions can explain both the levels of α-enrichment and the excess of low-mass stars as a function of stellar mass, observed for local spheroidal galaxies. As a preliminary test of our scenario, we use idealized star formation histories to estimate the mean IMF shape for galaxies of different z = 0 stellar mass. We show that the fraction of low-mass stars as a function of galaxy stellar mass predicted by these mean IMFs agrees with the values derived from high-resolution spectroscopic surveys.