The N2 index ([N II] λ6584/Hα) is used to determine emission line galaxy metallicities at all redshifts, including high redshift, where galaxies tend to be metal-poor. The initial aim of this work was to improve the calibrations used to infer oxygen abundance from N2 by employing updated low-metallicity galaxy databases. We compare N2 and the metallicity determined using the direct method for the set of extremely metal-poor galaxies compiled by Morales-Luis et al. To our surprise, the oxygen abundance presents a tendency to be constant with N2, with a very large scatter. Consequently, we find that the existing N2 calibrators overestimate the oxygen abundance for most low-metallicity galaxies, and can therefore only be used to set upper limits to the true metallicity in low-metallicity galaxies. An explicit expression for this limit is given. In addition, we try to explain the observed scatter using photoionization models. It is mostly due to the different evolutionary state of the H II regions producing the emission lines, but it also arises due to differences in N/O among the galaxies.