Context. The physical conditions in stellar atmospheres, such as temperature, velocity, and magnetic fields, can be obtained by interpreting solar spectro-polarimetric observations. Traditional inversion codes, while successful, are computationally demanding, however, especially for lines whose formation is complex and dictated by nonlocal thermodynamical equilibrium effects. The necessity of faster alternatives, in particular, with the increasing volume of data from large solar telescopes, has motivated the emergence of machine-learning solutions. Aims. This paper introduces an approach to the inversion and synthesis of Stokes profiles that is inspired by neural machine translation. Our aim is to develop a generative model that treats Stokes profiles and atmospheric models as two distinct languages that encode the same physical reality. We built a model that learned how to translate between them and also provides estimates of the uncertainty. Methods. We employed a tokenization strategy for the Stokes parameters and model atmospheres that was learned using vector-quantized variational autoencoders (VQ-VAE). This neural model was used to compress the data into a lower-dimensionality form. The core of our inversion code uses a transformer encoder-decoder architecture, similar to those used in natural language processing, to perform the translation between these tokenized representations. The model was trained on a comprehensive database of synthetic Stokes profiles derived from perturbations to various semi-empirical solar atmospheric models to ensure a wide range of expected solar physical conditions. Results. The method effectively reconstructed atmospheric models from observed Stokes profiles and showed better constrained models within the region of sensitivity of the considered spectral lines. The latent representation induced by the VQ-VAE helped us to accelerate the inversion by compressing the length of the Stokes profiles and model atmospheres. Additionally, it helped us to regularize the solution by reducing the chances of obtaining unphysical models. As a final and crucial advantage, the method we describe provides the generative nature of our model, which naturally yields an estimate of the uncertainty in the solution.