The relatively recent insight that energy input from supermassive black holes (BHs) can have a substantial effect on the star formation rates (SFRs) of galaxies motivates us to examine the effects of BH feedback on the scale of galaxy groups. At present, groups contain most of the galaxies and a significant fraction of the overall baryon content of the Universe and, along with massive clusters, they represent the only systems for which it is possible to measure both the stellar and gaseous baryonic components directly. To explore the effects of BH feedback on groups, we analyse two high-resolution cosmological hydrodynamic simulations from the OverWhelmingly Large Simulations (OWLS) project. While both include galactic winds driven by supernovae, only one of the models includes feedback from accreting BHs. We compare the properties of the simulated galaxy groups to a wide range of observational data, including the entropy and temperature profiles of the intragroup medium, hot gas mass fractions, the luminosity-temperature and mass-temperature scaling relations, the K-band luminosity of the group and its central brightest galaxy (CBG), SFRs and ages of the CBG, and gas- and stellar-phase metallicities. Both runs yield entropy distributions similar to the data, while the run without active galactic nucleus (AGN) feedback yields highly peaked temperature profiles, in discord with the observations. Energy input from supermassive BHs significantly reduces the gas mass fractions of galaxy groups with masses less than a few × 1014 Msolar, yielding a gas mass fraction and X-ray luminosity scaling with system temperature that is in excellent agreement with the data, although the detailed scatter in the L-T relation is not quite correct. The run without AGN feedback suffers from the well-known overcooling problem - the resulting stellar mass fractions are several times larger than observed and present-day cooling flows operate uninhibitedly. By contrast, the run that includes BH feedback yields stellar mass fractions, SFRs and stellar age distributions in excellent agreement with current estimates, thus resolving the long-standing `cooling crisis' of simulations on the scale of groups. Both runs yield very similar gas-phase metal abundance profiles that match X-ray measurements, but they predict very different stellar metallicities. Based on the above, galaxy groups provide a compelling case that feedback from supermassive BHs is a crucial ingredient in the formation of massive galaxies.