We have analyzed bibliographical observational data and theoretical predictions, in order to probe the environment in which extremely metal-poor dwarf galaxies (XMPs) reside. We have assessed the H i component and its relation to the optical galaxy, the cosmic web type (voids, sheets, filaments and knots), the overdensity parameter and analyzed the nearest galaxy neighbors. The aim is to understand the role of interactions and cosmological accretion flows in the XMP observational properties, particularly the triggering and feeding of the star formation. We find that XMPs behave similarly to Blue Compact Dwarfs; they preferably populate low-density environments in the local universe: ∼60% occupy underdense regions, and ∼75% reside in voids and sheets. This is more extreme than the distribution of irregular galaxies, and in contrast to those regions preferred by elliptical galaxies (knots and filaments). We further find results consistent with previous observations; while the environment does determine the fraction of a certain galaxy type, it does not determine the overall observational properties. With the exception of five documented cases (four sources with companions and one recent merger), XMPs do not generally show signatures of major mergers and interactions; we find only one XMP with a companion galaxy within a distance of 100 kpc, and the H i gas in XMPs is typically well-behaved, demonstrating asymmetries mostly in the outskirts. We conclude that metal-poor accretion flows may be driving the XMP evolution. Such cosmological accretion could explain all the major XMP observational properties: isolation, lack of interaction/merger signatures, asymmetric optical morphology, large amounts of unsettled, metal-poor H i gas, metallicity inhomogeneities, and large specific star formation.