We outline the science prospects for gamma-ray bursts (GRBs) with the Cherenkov Telescope Array (CTA), the next-generation ground-based gamma-ray observatory operating at energies above few tens of GeV. With its low energy threshold, large effective area and rapid slewing capabilities, CTA will be able to measure the spectra and variability of GRBs at multi-GeV energies with unprecedented photon statistics, and thereby break new ground in elucidating the physics of GRBs, which is still poorly understood. Such measurements will also provide crucial diagnostics of ultra-high-energy cosmic ray and neutrino production in GRBs, advance observational cosmology by probing the high-redshift extragalactic background light and intergalactic magnetic fields, and contribute to fundamental physics by testing Lorentz invariance violation with high precision. Aiming to quantify these goals, we present some simulated observations of GRB spectra and light curves, together with estimates of their detection rates with CTA. Although the expected detection rate is modest, of order a few GRBs per year, hundreds or more high-energy photons per burst may be attainable once they are detected. We also address various issues related to following up alerts from satellites and other facilities with CTA, as well as follow-up observations at other wavelengths. The possibility of discovering and observing GRBs from their onset including short GRBs during a wide-field survey mode is also briefly discussed.