Using Herschel Photodetector Array Camera (PACS) and Spectral and Photometric Imaging Receiver (SPIRE) observations of Lockman Hole-North and Great Observatories Origins Deep Survey-North (GOODS-N) as part of the Herschel Multi-tiered Extragalactic Survey (HerMES) project, we explore the far-infrared (IR) properties of a sample of mid-IR-selected starburst-dominated ultraluminous infrared galaxies (ULIRGs) at z ~ 2. The selection of the sample is based on the detection of the stellar bump that appears in the spectral energy distribution of star-forming galaxies at 1.6μm. We derive robust estimates of infrared luminosities (LIR) and dust temperatures (Td) of the population and find that while the luminosities in our sample span less than an order of magnitude (12.24 <= log(LIR/Lsolar) <= 12.94), they cover a wide range of dust temperatures (25 <= Td <= 62K). Galaxies in our sample range from those that are as cold as high-z submillimetre galaxies (SMGs) to those that are as warm as optically faint radio galaxies (OFRGs) and local ULIRGs. Nevertheless, our sample has median Td = 42.3K, filling the gap between SMGs and OFRGs, bridging the two populations. We demonstrate that a significant fraction of our sample would be missed from ground-based (sub)mm surveys (850-1200μm), showing that the latter introduce a bias towards the detection of colder sources. We conclude that Herschel observations confirm the existence of high-z ULIRGs warmer than SMGs, show that the mid-IR selection of high-z ULIRGs is not Td dependent, reveal a large dispersion in Td of high-z ULIRGs and provide the means to characterize the bulk of the ULIRG population, free from selection biases introduced by ground-based (sub)mm surveys.