Cosmic history has witnessed the lives and deaths of multiple generations of massive stars, all of them invigorating their host galaxies with ionizing photons, kinetic energy, fresh material, and stellar-mass black holes. Ubiquitous engines as they are, astrophysics needs a good understanding of their formation, evolution, properties and yields throughout the history of the Universe, and with decreasing metal content mimicking the environment at the earliest epochs. Ultimately, a physical model that could be extrapolated to zero metallicity would enable tackling long-standing questions such as "What did the first, very massive stars of the Universe look like?" or "What was their role in the re-ionization of the Universe?" Yet, most of our knowledge of metal-poor massive stars is drawn from one single point in metallicity. Massive stars in the Small Magellanic Cloud (SMC, ∼1/5Z⊙ ) currently serve as templates for low-metallicity objects in the early Universe, even though significant differences with respect to massive stars with poorer metal content have been reported. This White Paper summarizes the current knowledge on extremely (sub-SMC) metal poor massive stars, highlighting the most outstanding open questions and the need to supersede the SMC as standard. A new paradigm can be built from nearby extremely metal-poor galaxies that make a new metallicity ladder, but massive stars in these galaxies are out of reach to current observational facilities. Such a task would require an L-size mission, consisting of a 10m-class space telescope operating in the optical and the ultraviolet ranges. Alternatively, we propose that ESA unites efforts with NASA to make the LUVOIR mission concept a reality, thus continuing the successful partnership that made the Hubble Space Telescope one of the greatest observatories of all time.