We study radiative transfer through a weakly polarizing medium, i.e., a medium in which the degree of polarization of the absorbed, retarded, and emitted light is always weak. In this case, the general radiative transfer equation for the Stokes parameters yields very simple formal solutions. The intensity does not depend on the polarization, and the other Stokes parameters are uncoupled from each other. It is shown how this simplified radiative transfer equation holds in many realistic cases relevant for solar and stellar magnetometry. It can be applied whenever the weak magnetic field approximation works, i.e., for weakly split lines. In addition, it handles weak spectral lines, structures with complex magnetic topology, chromospheric lines formed under non-LTE conditions, etc. The merits of the approximation, which we call the weakly polarizing medium (WPM) approximation, are illustrated by means of several LTE and non-LTE line syntheses in realistic solar model atmospheres. The WPM approximation should be useful in planning and understanding measurements based on polarization. It simplifies the relationship between the observed polarization and the physical structure that one tries to retrieve. The approximation may also be used in numerical problems requiring extensive polarized radiative transfer (inversion codes, syntheses of stellar spectra, self-consistent multilevel non-LTE Zeeman line transfer with atomic polarization, etc.).