Bright points (BPs) visible in the G band at 430 nm are commonly used as tracers of magnetic fields, indicating the location of kG flux concentrations. To study the actual magnetic properties of G-band BPs, we took observations in 2003 and 2005, employing simultaneously a speckle setup in the G band and vector spectropolarimetry to derive the magnetic field vector. From the analysis of the co-aligned polarimetric data we find that the BPs show a broad range of field strengths, magnetic fluxes, and field inclinations. Many G-band BPs are not co-spatial with the central part of the nearby flux concentrations. Even at the small heliocentric angle of only 12°, the BPs appear projected on adjacent granules, whereas the magnetic field is concentrated in the intergranular lanes. Our findings support the view that the G-band BPs are a result of the "hot wall effect". The downward shift of the optical depth scale in the presence of magnetic fields allows to see deeper and hotter layers in the hot granules next to the field concentrations, where CH dissociates. Thus, information drawn from imaging observations of BPs has limited use to investigate the actual magnetic field structure, when the BPs are not co-spatial with the central part of the flux concentrations.