Now almost 70 years since its introduction, the effective or half-light radius has become a very popular choice for characterising galaxy size. However, the effective radius measures the concentration of light within galaxies and thus does not capture our intuitive definition of size which is related to the edge or boundary of objects. For this reason, we aim to demonstrate the undesirable consequence of using the effective radius to draw conclusions about the nature of faint ultra-diffuse galaxies (UDGs) when compared to dwarfs and Milky Way-like galaxies. Instead of the effective radius, we use a measure of galaxy size based on the location of the gas density threshold required for star formation. Compared to the effective radius, this physically motivated definition places the sizes much closer to the boundary of a galaxy. Therefore, considering the sizes and stellar mass density profiles of UDGs and regular dwarfs, we find that the UDGs have sizes that are within the size range of dwarfs. We also show that currently known UDGs do not have sizes comparable to Milky Way-like objects. We find that, on average, UDGs are ten times smaller in extension than Milky Way-like galaxies. These results show that the use of size estimators sensitive to the concentration of light can lead to misleading results.