The ability of any Machine Learning method to classify the spectra of galaxies depending on the properties of the stellar component rests on the information content of the data. The well-known degeneracies found in population synthesis models suggest this information might be so entangled as to challenge the most sophisticated Deep Learning approaches. This contribution focuses on the traditional definition of entropy to explore this problem from a fundamental viewpoint. We find that the information content – when interpreting the spectrum as a probability distribution function – is reduced to a few spectral intervals that are strongly correlated. Dimensionality reduction via PCA suggests the standard 4000Å break strength and Balmer absorption are the two most informative regions in the analysis of galaxy spectra.