Galactic synchrotron emission exhibits large angular scale features known as radio spurs and loops. Determining the physical size of these structures is important for understanding the local interstellar structure and for modeling the Galactic magnetic field. However, the distance to these structures is either under debate or entirely unknown. We revisit a classical method of finding the location of radio spurs by comparing optical polarization angles with those of synchrotron emission as a function of distance. We consider three tracers of the magnetic field: stellar polarization, polarized synchrotron radio emission, and polarized thermal dust emission. We employ archival measurements of optical starlight polarization and Gaia distances and construct a new map of polarized synchrotron emission from WMAP and Planck data. We confirm that synchrotron, dust emission, and stellar polarization angles all show a statistically significant alignment at high Galactic latitude. We obtain distance limits to three regions toward Loop I of 112 ± 17 pc, 135 ± 20 pc, and <105 pc. Our results strongly suggest that the polarized synchrotron emission toward the North Polar Spur at b > 30° is local. This is consistent with the conclusions of earlier work based on stellar polarization and extinction, but in stark contrast with the Galactic center origin recently revisited on the basis of X-ray data. We also obtain a distance measurement toward part of Loop IV (180 ± 15 pc) and find evidence that its synchrotron emission arises from chance overlap of structures located at different distances. Future optical polarization surveys will allow the expansion of this analysis to other radio spurs.