We have obtained spectra with the 10-m Keck telescope of a sample of 24 galaxies having colors consistent with star-forming galaxies at redshifts 2 ≲ z ≲ 4.5 in the Hubble Deep Field (HDF). Eleven of these galaxies are confirmed to be at high redshift (zmed=3.0). The spectra of the confirmed high-redshift galaxies show a diversity of features, including weak Lyalpha emission, strong Lyalpha breaks or damped Lyalpha absorption profiles, and the stellar and interstellar rest-UV absorption lines common to local starburst galaxies and high-redshift star-forming galaxies reported recently by others. The galaxies are small but luminous, with half-light radii 1.8 < r_{1/2} < 6.5 h50(-1) kpc and absolute magnitudes -21.5 > M_B > -23, and show a wide range of morphologies. The observed number density of high-redshift galaxies in the HDF is comparable to estimates of the volume density of local galaxies brighter than L*. We suggest that many may be low-mass objects in an early burst phase of star formation and that they will soon merge with other similar objects to build massive galaxies, or else consume or expel their gas and thus quench their star formation.