The two sources AGC 226178 and NGVS 3543, an extremely faint, clumpy, blue stellar system and a low surface brightness dwarf spheroidal, are adjacent systems in the direction of the Virgo cluster. Both have been studied in detail previously, with it being suggested that they are unrelated normal dwarf galaxies or that NGVS 3543 recently lost its gas through ram pressure stripping and AGC 226178 formed from this stripped gas. However, with Hubble Space Telescope Advanced Camera for Surveys imaging, we demonstrate that the stellar population of NGVS 3543 is inconsistent with being at the distance of the Virgo cluster and that it is likely a foreground object at approximately 10 Mpc, whereas the stellar population of AGC 226178 is consistent with it being a very young (10-100 Myr) object in the Virgo cluster. Through a reanalysis of the original ALFALFA H I detection, we show that AGC 226178 likely formed from gas stripped from the nearby dwarf galaxy VCC 2034, a hypothesis strengthened by the high metallicity measured with MUSE VLT observations. However, it is unclear whether ram pressure or a tidal interaction is responsible for stripping the gas. Object AGC 226178 is one of at least five similar objects now known toward Virgo. These objects are all young and unlikely to remain visible for over ~500 Myr, suggesting that they are continually produced in the cluster.