Context. Gamma-ray bursts (GRBs) can provide information about star formation at high redshifts. Even in the absence of a bright optical/near-infrared/radio afterglow, the high detection rate of X-ray afterglows by Swift/XRT and its localization precision of 2-3 arcsec facilitates the identification and the study of GRB host galaxies. Aims: We focus on the search for the host galaxies of 17 bursts with arcsec-sized XRT error circles but no detected long-wavelength afterglow, in spite of their deep and rapid follow-up observations. Three of these events can also be classified as truly dark bursts, i.e., the observed upper limit on the optical flux of the afterglow was less than expected based on the measured X-ray flux. Our goals are to identify the GRB host galaxy candidates and characterize their phenomenological parameters. Methods: Our study is based on deep RC and Ks-band observations performed with FORS1, FORS2, VIMOS, ISAAC, and HAWK-I at the ESO/VLT, partly supported by observations with the seven-channel imager GROND at the 2.2-m telescope on La Silla, and supplemented by observations with NEWFIRM at the 4-m telescope on Kitt Peak. To be conservative, we searched for host galaxy candidates within an area of twice the radius of each associated 90% c.l. Swift/XRT error circle. Results: For 15 of the 17 bursts, we find at least one galaxy within the searching area, and in the remaining two cases only a deep upper limit to RC and Ks can be provided. In seven cases, we discover extremely red objects in the error circles, at least four of which might be dust-enshrouded galaxies. The most remarkable case is the host of GRB 080207, which has a color of (RC - Ks)AB ~ 4.7 mag, and is one of the reddest galaxies ever associated with a GRB. As a by-product of our study we identify the optical afterglow of GRB 070517. Conclusions: Only a minority of optically dim afterglows are due to Lyman dropout (≲ 1/3). Extinction by dust in the host galaxies might explain all other events. Thereby, a seemingly non-negligible fraction of these hosts are globally dust-enshrouded, extremely red galaxies. This suggests that at least a fraction of GRB afterglows trace a subpopulation of massive starburst galaxies, which are markedly different from the main body of the GRB host galaxy population, namely the blue, subluminous, compact galaxies. Based on observations collected at the Very Large Telescope of the European Southern Observatory, Paranal, Chile (ESO programmes 381.A-0647, 383.A-0399, 384.A-0414; PI: S. Klose; 081.D-0739, PI: A. Rossi, and 086.A-0533, PI: T. Krühler), GROND (PI: J. Greiner), and the Kitt Peak National Observatory (Program ID 2008B-0070; PI: A. C. Updike). Other observations are obtained from the ESO/ST-ECF Science Archive Facility.Appendix A is available in electronic form at http://www.aanda.org