Our knowledge of galaxy formation and evolution has incredibly progressed through multi-wavelength observational constraints of the interstellar medium (ISM) of galaxies at all cosmic epochs. However, little is known about the physical properties of the more diffuse and lower surface brightness reservoir of gas and dust that extends beyond ISM scales and fills dark matter haloes of galaxies up to their virial radii, the circumgalactic medium (CGM). New theoretical studies increasingly stress the relevance of the latter for understanding the feedback and feeding mechanisms that shape galaxies across cosmic times, whose cumulative effects leave clear imprints into the CGM. Recent studies are showing that a – so far unconstrained – fraction of the CGM mass may reside in the cold (T < 104 K) molecular and atomic phase, especially in high-redshift dense environments. These gas phases, together with the warmer ionised phase, can be studied in galaxies from z ∼ 0 to z ∼ 10 through bright far-infrared and sub-millimeter emission lines such as [C ii] 158µm, [O iii] 88 µm, [C I] 609µm, [C i] 370µm, and the rotational transitions of CO. Imaging such hidden cold CGM can lead to a breakthrough in galaxy evolution studies but requires a new facility with the specifications of the proposed Atacama Large Aperture Submillimeter Telescope (AtLAST). In this paper, we use theoretical and empirical arguments to motivate future ambitious CGM observations with AtLAST and describe the technical requirements needed for the telescope and its instrumentation to perform such science.