The Instituto de Astrofísica de Canarias (IAC) has led the development of a new numerical procedure that allows to reproduce in a few seconds with Big Data and machine learning techniques the intergalactic medium obtained from a cosmological simulation of 100,000 hours of computation. Thanks to this algorithm, called Hydro-BAM, researchers have been able to exploit the hierarchy in the relationship between the properties of dark matter, ionized gas and intergalactic neutral hydrogen, ingredients that make up the large-scale structure of our universe. The research has also made it possible to reproduce with high precision the so-called "Lyman-alpha forests", a pattern of lines in the spectra of distant galaxies and quasars that is produced when the light emitted by these objects is absorbed, along its path, by clouds of hydrogen gas. The analysis of these forests is fundamental to advance our understanding of the Universe as a whole. The study has led to the publication of two papers in The Astrophysical Journal.

Current observations seem to indicate that everything in our universe is dominated by dark matter and dark energy, much more abundant than conventional or baryonic matter. This matter, the one we can see, accounts for only 5% of the entire mass of the Universe. In contrast, dark matter, invisible to our eyes, makes up approximately 27% of our immense cosmos. The remaining 68% is composed of dark energy, which is not only responsible for the Universe expanding, but also for its constant acceleration.

The standard cosmological model assumes that the organization of the Universe on its largest scales depends on the interaction of these ingredients. Indeed, current state-of-the-art numerical simulations are beginning to provide realistic modeling of these processes. However, a large number of uncertainties remain. 

To obtain reliable theoretical predictions, scientists need to perform large sets of numerical simulations that cover a large cosmological volume and are based on different possible models that include all relevant physical processes. These "virtual universes" serve as test beds for the study of cosmology. However, the simulations are computationally very expensive and current computing facilities only allow to explore small cosmic volumes compared to the volumes covered by current and future observational campaigns.

Big Data and AI to decode the Universe

Therefore, a collaboration between a team from the Instituto de Astrofísica de Canarias (IAC), led by Francisco-Shu Kitaura, and another from Osaka University, led by Kentaro Nagamine, have developed new strategies that make it possible to recreate, in a detailed and fast way, computational models on the formation and evolution of the Universe. 

"We are making a special effort to develop machine learning techniques to speed up the whole process, save computational costs and efficiently run many of these simulations," Francesco Sinigaglia, a joint PhD student at the University of La Laguna (Tenerife, Spain) and the IAC and the University of Padua (Italy), first author of both publications. 

Specifically, the IAC team has developed a novel algorithm, called Hydro-BAM, which combines advanced concepts of probability theory, machine learning and cosmology. This algorithm has made it possible to obtain very accurate predictions in just a few tens of seconds, which is equivalent to the costly hydrodynamic simulation, lasting approximately 100,000 hours on a supercomputer, carried out by the Osaka team and used to train Hydro-BAM. "The algorithm is about 100,000 lines of code written at the IAC resulting from the effort of years of work by a few researchers, about the same number as the first version of Photoshop," notes Francisco-Shu Kitaura, researcher at the IAC.

"The goal of these studies is to refine our understanding of the large-scale structure of the Universe and to infer information about its evolution over cosmic time by modeling and observing baryonic quantities," stresses Andrés Balaguera Antolínez, IAC researcher and one of the main developers of the Hydro-BAM code. "Our methods aim to reproduce the observed Universe through a detailed evaluation of the different and complex statistical links between the three-dimensional distribution of dark matter and visible matter such as galaxies and intergalactic gas."

Gas trees revealing the cosmic forest

Using this new computational procedure, the researchers addressed the connection to the observable Universe.  "We have performed an exhaustive post-processing analysis of our hydrodynamic simulations by putting millions of virtual observers to model the Lyman-alpha forest observed in the absorption of quasar lines of sight," describes Ikkoh Shimuzu, formerly of Osaka University (now at Shikoku Gakuin University). This pattern is produced when "trees" of hydrogen gas, scattered throughout the Universe, absorb the light emitted by these distant objects. In this way, scientists can see distinct absorption lines corresponding to clouds at different distances and thus showing different ages of the Universe, as well as providing information about the intergalactic medium.

"The breakthrough came when we understood that the connections between the quantities of intergalactic gas, dark matter and neutral hydrogen that we were trying to model are well organized in a hierarchical way," says Sinigaglia. "The ionized gas has a distribution in space very similar to that of the dark matter and the neutral hydrogen is determined by the distribution of the ionized gas; moreover, the joint distribution of the ionized gas and neutral hydrogen gives us information about the thermal state of the gas and allows us to predict the absorption flux of the Lyman-alpha forest," he concludes.

"Our papers in this field are having a great impact on the scientific community and we have been contacted by world-class groups," says Kitaura. Despite their success, the authors consider that the research is just beginning and plan to produce thousands of simulated universes including baryonic physics, which should allow a comprehensive analysis of data from galaxy surveys such as DESI, WEAVE-JPAS and the Subaru PFS project. In particular, the outcome of this research will allow scientists to perform an unprecedented analysis of massive Lyman-alpha forest data sets, which allows us to address the possible tensions of cosmological models obtained from different observational probes.

Article: “Mapping the Three-dimensional Lyα Forest Large-scale Structure in Real and Redshift Space”, The Astrophysical Journal, 2022 March, Volume 927, Number 2. DOI: https://doi.org/10.3847/1538-4357/ac5112

Article: “The Bias from Hydrodynamic Simulations: Mapping Baryon Physics onto Dark Matter Fields”, The Astrophysical Journal, 2021 November, Volume 921, Number 1. DOI: https://doi.org/10.3847/1538-4357/ac158b

Contact at the IAC:
Francisco-Shu Kitaura, fkitaura [at] iac.es (fkitaura[at]iac[dot]es)
Andrés Balaguera-Antolínez, balaguera [at] iac.es (balaguera[at]iac[dot]es)

Contact at Osaka University:
Kentaro Nagamine, kn [at] astro-osaka.jp